diff --git a/BreakingChanges.md b/BreakingChanges.md new file mode 100644 index 00000000..e2aaab4e --- /dev/null +++ b/BreakingChanges.md @@ -0,0 +1,34 @@ +**Note: This BreakingChanges.md file is deprecated after version 0.15.0-preview, please refer to the BreakingChange.md in each package for future change logs.** + +Tracking Breaking Changes in 0.14.0-preview + +QUEUE +* The return type of `Azure::Storage::Queue::create_message` is changed from `nil` to an `Azure::Storage::Queue::Message` object. + +Tracking Breaking Changes in 0.13.0-preview + +TABLE +* The return type `Azure::Service::EnumerationResult` of `query_tables` has a changed structure. Now the `'updated'` will not be contained, and is flattened to a structure in the form of `{ {"TableName" => "tableone"}, {"TableName" => "tabletwo"}, {"TableName" => "tablethree"}}`. +* The `Azure::Storage::Table::Entity` does not contain `:table` and `updated` anymore. The updated time can be found in `:properties`. +* The return type of `get_table` is changed to a Hash that contains full metadata returned from the server when query the table. +* The method `Azure::Storage::Table::EdmType::unserialize_query_value` is renamed to `deserialize_value`. + +Tracking Breaking Changes in 0.11.0-preview + +ALL +* `Azure::Storage::Core::Auth::SharedAccessSignature.generate` is renamed to `generate_service_sas_token`. +* `Azure::Storage::Core::Auth::SharedAccessSignature.signed_uri` requires `use_account_sas` as the second parameter. +* Removed the support for the Shared Key Lite. + +Tracking Breaking Changes in 0.10.0-preview + +ALL +* Require "azure-storage" instead of "azure_storage". + +BLOB +* The `create_blob_pages` method is renamed to `put_blob_pages`. +* The `create_blob_block` method is renamed to `put_blob_block`. +* The `acquire_lease` method is renamed to `acquire_blob_lease`. +* The `renew_lease` method is renamed to `renew_blob_lease`. +* The `release_lease` method is renamed to `release_blob_lease`. +* The `break_lease` method is renamed to `break_blob_lease`. diff --git a/blob/CONTRIBUTING.md b/CONTRIBUTING.md similarity index 90% rename from blob/CONTRIBUTING.md rename to CONTRIBUTING.md index 9db89891..308276f2 100644 --- a/blob/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -3,7 +3,7 @@ If you intend to contribute to the project, please make sure you've followed the The Azure Storage development team uses Visual Studio Code so instructions will be tailored to that preference. However, any preferred IDE or other toolset should be usable. ### Install -* Ruby 1.9.3, 2.0, 2.1 or 2.2. +* Ruby 1.9.3 to 2.5.0. * [Visual Studio Code](https://code.visualstudio.com/) ### Development Environment Setup @@ -43,7 +43,7 @@ You can use the following commands to run: * All the tests: ``rake test``. **This will run integration tests if you have .env file or env vars setup** * Run storage suite of tests: ``rake test:unit``, ``rake test:integration`` -* One particular test file: ``ruby -I"lib:test" ""`` +* One particular test file: ``ruby -I".\blob\lib;.\common\lib;.\table\lib;.\queue\lib;.\file\lib;test" ""`` ### Testing Features As you develop a feature, you'll need to write tests to ensure quality. Your changes should be covered by both unit tests and integration tests. You should also run existing tests related to your change to address any unexpected breaks. @@ -59,11 +59,11 @@ The following are the minimum requirements for any pull request that must be met * You should strive to mimic the style with which we have written the library * Clean, well-commented, well-designed code * Try to limit the number of commits for a feature to 1-2. If you end up having too many we may ask you to squash your changes into fewer commits. -* [ChangeLog.md](ChangeLog.md) needs to be updated describing the new change +* ChangeLog.md in service module folders needs to be updated describing the new change * Thoroughly test your feature ### Branching Policy -Changes should be based on the **dev** branch, not master as master is considered publicly released code. Each breaking change should be recorded in [BreakingChanges.md](BreakingChanges.md). +Changes should be based on the **dev** branch, not master as master is considered publicly released code. Each breaking change should be recorded in the corresponding BreakingChanges.md in service module folders. Note that [BreakingChanges.md](BreakingChanges.md) is deprecated after GA release. ### Adding Features for All Platforms We strive to release each new feature for each of our environments at the same time. Therefore, we ask that all contributions be written for Ruby 1.9.3 and later. diff --git a/ChangeLog.md b/ChangeLog.md new file mode 100644 index 00000000..48699f0e --- /dev/null +++ b/ChangeLog.md @@ -0,0 +1,198 @@ +**Note: This changelog is deprecated after version 0.15.0-preview, please refer to the ChangeLog.md in each package for future change logs.** + +2017.11 - version 0.15.0-preview + +ALL +* Added the support for the location mode in the API options. +* Added the support for retrying according to the location mode. + +BLOB +* Added the support for retrieving statistics related to replication for the Blob service. +* Added the support for anonymous read access of public containers. +* Added full lease ID header support for following methods: + - Azure::Storage::Blob::BlobService::get_container_properties + - Azure::Storage::Blob::BlobService::get_container_metadata + - Azure::Storage::Blob::BlobService::get_container_acl + - Azure::Storage::Blob::BlobService::set_container_metadata + - Azure::Storage::Blob::BlobService::set_container_acl + - Azure::Storage::Blob::BlobService::delete_container + - Azure::Storage::Blob::BlobService::create_block_blob + - Azure::Storage::Blob::BlobService::create_page_blob + - Azure::Storage::Blob::BlobService::create_append_blob + - Azure::Storage::Blob::BlobService::get_blob + - Azure::Storage::Blob::BlobService::get_blob_properties + - Azure::Storage::Blob::BlobService::set_blob_properties + - Azure::Storage::Blob::BlobService::get_blob_metadata + - Azure::Storage::Blob::BlobService::set_blob_metadata + - Azure::Storage::Blob::BlobService::create_blob_snapshot + - Azure::Storage::Blob::BlobService::copy_blob + - Azure::Storage::Blob::BlobService::copy_blob_from_uri + - Azure::Storage::Blob::BlobService::delete_blob + - Azure::Storage::Blob::BlobService::put_blob_block + - Azure::Storage::Blob::BlobService::commit_blob_blocks + - Azure::Storage::Blob::BlobService::list_blob_blocks + - Azure::Storage::Blob::BlobService::put_blob_pages + - Azure::Storage::Blob::BlobService::list_page_blob_ranges + - Azure::Storage::Blob::BlobService::incremental_copy_blob + - Azure::Storage::Blob::BlobService::append_blob_block + +Queue +* Added the support for retrieving statistics related to replication for the Queue service. + +Table +* Added the support for retrieving statistics related to replication for the Table service. + +2017.09 - version 0.14.0-preview + +ALL +* Added configuration file for Rubocop and auto-resolved coding style issue. + +BLOB +* The `Azure::Storage::Blob::list_page_blob_ranges` API now accepts `:previous_snapshot` as an optional parameter, that specifies that the response returns pages that have been updated or cleared since the snapshot specified by `:previous_snapshot` was taken. +* The `Azure::Storage::Blob::Blob` object now has an attribute `:encrypted` showing if the blob or blob related request has been encrypted. +* The `Azure::Storage::Blob::BlobService::list_containers` and `Azure::Storage::Blob::BlobService::get_container_properties` will now also return public access level for each container. +* The stored Content-MD5 property is now returned when requesting a range of a blob. Previously this was only returned for full blob downloads. `Azure::Storage::Blob::Blob.properties[:content_md5]` will always hold the stored Content_MD5 property, and `Azure::Storage::Blob::Blob.properties[:range_md5]` will always represent the MD5 for the content returned from the server. +* Added an API `Azure::Storage::Blob::BlobService::incremental_copy_blob` to support [incremental copy](https://docs.microsoft.com/en-us/rest/api/storageservices/incremental-copy-blob) for page blob snapshots. + +FILE +* The stored Content-MD5 property is now returned when requesting a range of a file. Previously this was only returned for full file downloads. `Azure::Storage::File::File.properties[:content_md5]` will always hold the stored Content_MD5 property, and `Azure::Storage::File::File.properties[:range_md5]` will always represent the MD5 for the content returned from the server. + +QUEUE +* The return type of `Azure::Storage::Queue::create_message` is changed from `nil` to an `Azure::Storage::Queue::Message` object. + +FILE +* The API `Azure::Storage::File::list_directories_and_files` now also accepts `:prefix` as an optional parameter. The return value will be filtered with the specified prefix if set. + +2017.09 - version 0.13.0-preview + +ALL +* Removed Nokogiri from Gemfile because it causes bundler fail to install azure-storage. Added it back to runtime dependency and explicitly require user to install the correct version of Nokogiri in README.md. +* Service version is upgraded to 2016-05-31. + +BLOB +* Block size can now be up to 100MB. + +TABLE +* The return type `Azure::Service::EnumerationResult` of `query_tables` has a changed structure. Now the `'updated'` will not be contained, and is flattened to a structure in the form of `{ {"TableName" => "tableone"}, {"TableName" => "tabletwo"}, {"TableName" => "tablethree"}}`. +* The `Azure::Storage::Table::Entity` does not contain `:table` and `updated` anymore. The updated time can be found in `:properties`. +* The return type of `get_table` is changed to a Hash that contains full metadata returned from the server when query the table. +* The method `Azure::Storage::Table::EdmType::unserialize_query_value` is renamed to `deserialize_value`. + +2017.08 - version 0.12.3-preview + +ALL +* Added Nokogiri as a gem into Gemfile, resolving an issue where bundler failed to recognize that the dependency exists after installation. + +2017.08 - version 0.12.2-preview + +ALL +* Removed Nokogiri as a dependency to resolve conflict version caused by azure-core also depending on Nokogiri. + +2017.04 - version 0.12.1-preview + +ALL +* Relaxed constraint on Nokogiri version dependency to allow Nokogiri 1.7.x for Ruby 2.1 and later, but preserving support for Ruby 1.9.3. (Note that this may require updating to Bundler 1.13 or later if you're using Ruby 1.9 or 2.0.) + +2017.02 - version 0.12.0-preview + +ALL +* Fixed the issue where `should_retry?` in the retry_filter.rb overwrites the result from derived `apply_retry_policy`. [#76](https://github.com/Azure/azure-storage-ruby/issues/76) +* Fixed the issue where `Azure::Storage::Client.create_from_connection_string` throws an exception. [#77](https://github.com/Azure/azure-storage-ruby/issues/77) +* Added the support for setting the "timeout" option in `get_service_properties` and `set_service_properties`. + +BLOB +* Added the metadata to the returning instance when creates a blob. +* Added `transactional_md5` to the options of `put_blob_pages`. + +FILE +* Added File Service support, targeting storage service version 2015-04-05. + +2016.12 - version 0.11.5-preview + +ALL +* Added the support for setting customer user agent. [#71](https://github.com/Azure/azure-storage-ruby/issues/71) +* Added the support for hooking in sending requests. + +2016.11 - version 0.11.4-preview + +ALL +* Removed the unnecessary dependencies. [#55](https://github.com/Azure/azure-storage-ruby/issues/55), [#67](https://github.com/Azure/azure-storage-ruby/issues/67) + +BLOB +* Fixed the issue when checking the content encoding. +* Fixed the wrong "Content-Encoding" value in the test cases. +* Fixed the issue where it cannot use the `create_block_blob` method with an IO/File object. [#61](https://github.com/Azure/azure-storage-ruby/issues/61) + +2016.10 - version 0.11.3-preview + +ALL +* Fixed an issue in retry policies. + +2016.10 - version 0.11.2-preview + +ALL +* Fixed the issue where it retries on HTTP 4xx errors. + +BLOB +* Fixed the issue of wrong "Content-Encoding". [#49](https://github.com/Azure/azure-storage-ruby/issues/49) + +2016.09 - version 0.11.1-preview + +ALL +* Added the support for setting the client request ID via the "request_id" parameter. +* Added the retry for the timeout errors. +* Added the retry for the connection reset error. + +BLOB +* Fixed the issue where "list_blobs" doesn't work when delimiter is specified. [#41](https://github.com/Azure/azure-storage-ruby/issues/41) + +2016.08 - version 0.11.0-preview + +ALL +* Added the support for the account shared access signature. +* Removed the support for the Shared Key Lite. + +BLOB +* Added the support for the "add" and "create" permissions in the blob service shared access signature. + +FILE +* Added the support for the "create" permission in the file service shared access signature. + +2016.06 - version 0.10.2-preview + +ALL +* Fixed the issue that cannot run against storage emulator on Windows. +* Fixed the issue that it doesn't run as a singleton when it calls Azure::Storage.setup. +* Updated to storage service version 2015-04-05. + +2016.05 - version 0.10.1-preview + +ALL +* Replaced the core module by the 'azure-core' gem. +* Stopped maintaining the support for Ruby 1.9. +* Added support for retry filters. + +2016.03 - version 0.10.0-preview + +ALL +* Separated out parts of Azure Storage previously found in the Azure SDK 0.7.0 to establish an independent release cycle. +* Supported Ruby 1.9.3, 2.0, 2.1 and 2.2. +* Updated to storage service version 2015-02-21. +* Fixed issue where previous query's parameters were used, causing authentication to fail (https://github.com/Azure/azure-sdk-for-ruby/issues/276). +* Fixed the issue that the Content-MD5 is overwritten when request body exists, regardless the input value. +* Refined the code for setting service properties to be compatible with the XML schema. + +BLOB +* Added support for container lease operations. +* Added support for changing the ID of an existing lease. +* Added support for copying from source blob URI. +* Added support for aborting copying a blob. +* Added support for creating an append blob. +* Added support for appending a block to an appendblob. +* Added support for the content disposition property of a blob. +* Added support for resizing a page blob. +* Added support for setting a page blob's sequence number. +* Fixed the issue where conditional headers for some APIs could not be set. +* Fixed the issue where the request fails when calling list_page_blob_ranges with start range and end range. +* Renamed create_blob_block to put_blob_block. +* Renamed create_blob_pages to put_blob_pages. diff --git a/README.md b/README.md new file mode 100644 index 00000000..45942b1e --- /dev/null +++ b/README.md @@ -0,0 +1,44 @@ +# Microsoft Azure Storage Client Library for Ruby + +* Master: [![Master Build Status](https://travis-ci.org/Azure/azure-storage-ruby.svg?branch=master)](https://travis-ci.org/Azure/azure-storage-ruby/branches) [![Coverage Status](https://coveralls.io/repos/github/Azure/azure-storage-ruby/badge.svg?branch=master)](https://coveralls.io/github/Azure/azure-storage-ruby?branch=master) +* Dev: [![Dev Build Status](https://travis-ci.org/Azure/azure-storage-ruby.svg?branch=dev)](https://travis-ci.org/Azure/azure-storage-ruby/branches) [![Coverage Status](https://coveralls.io/repos/github/Azure/azure-storage-ruby/badge.svg?branch=dev)](https://coveralls.io/github/Azure/azure-storage-ruby?branch=dev) + +This project provides Ruby packages that makes it easy to access and manage Microsoft Azure Storage Services. + +# Library Packages + +* [Blobs](https://github.com/azure/azure-storage-ruby/tree/master/blob) +* [Tables](https://github.com/azure/azure-storage-ruby/tree/master/table) +* [Queues](https://github.com/azure/azure-storage-ruby/tree/master/queue) +* [Files](https://github.com/azure/azure-storage-ruby/tree/master/file) + +Note: + +* x64 Ruby for Windows is known to have some compatibility issues. +* Each service gems depends on gem nokogiri. For Ruby version lower than 2.2, please install the compatible nokogiri before trying to install azure-storage. + +# Getting Started for Contributors + +If you would like to become an active contributor to this project please follow the instructions provided in [Azure Projects Contribution Guidelines](http://azure.github.io/guidelines/). +You can find more details for contributing in the [CONTRIBUTING.md](CONTRIBUTING.md). + +# Provide Feedback + +If you encounter any bugs with the library please file an issue in the [Issues](https://github.com/Azure/azure-storage-ruby/issues) section of the project. + +# Azure Storage SDKs and Tooling + +* [Azure Storage Client Library for .Net](http://github.com/azure/azure-storage-net) +* [Azure Storage Client Library for Java](http://github.com/azure/azure-storage-java) +* [Azure Storage Client Library for Node.js](http://github.com/azure/azure-storage-node) +* [Azure Storage Client Library for Python](http://github.com/azure/azure-storage-python) +* [Azure Storage Client Library for Ruby](http://github.com/azure/azure-storage-ruby) +* [Azure Storage Client Library for C++](http://github.com/azure/azure-storage-cpp) +* [Azure Storage Client Library for PHP](http://github.com/azure/azure-storage-php) +* [Azure Storage Blob Client Library for Go](https://github.com/Azure/azure-storage-blob-go) +* [Azure Storage Client Library for iOS](http://github.com/azure/azure-storage-ios) +* [Azure Storage Client Library for Android](http://github.com/azure/azure-storage-android) +* [Azure Storage Data Movement Library](https://github.com/Azure/azure-storage-net-data-movement) + +# Code of Conduct +This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments. diff --git a/blob/BreakingChanges.md b/blob/BreakingChanges.md index 94b645f9..f14368ed 100644 --- a/blob/BreakingChanges.md +++ b/blob/BreakingChanges.md @@ -1,32 +1,6 @@ -Tracking Breaking Changes in 0.14.0-preview +Tracking Breaking Changes in 1.0.0 -QUEUE -* The return type of `Azure::Storage::Queue::create_message` is changed from `nil` to an `Azure::Storage::Queue::Message` object. - -Tracking Breaking Changes in 0.13.0-preview - -TABLE -* The return type `Azure::Service::EnumerationResult` of `query_tables` has a changed structure. Now the `'updated'` will not be contained, and is flattened to a structure in the form of `{ {"TableName" => "tableone"}, {"TableName" => "tabletwo"}, {"TableName" => "tablethree"}}`. -* The `Azure::Storage::Table::Entity` does not contain `:table` and `updated` anymore. The updated time can be found in `:properties`. -* The return type of `get_table` is changed to a Hash that contains full metadata returned from the server when query the table. -* The method `Azure::Storage::Table::EdmType::unserialize_query_value` is renamed to `deserialize_value`. - -Tracking Breaking Changes in 0.11.0-preview - -ALL -* `Azure::Storage::Core::Auth::SharedAccessSignature.generate` is renamed to `generate_service_sas_token`. -* `Azure::Storage::Core::Auth::SharedAccessSignature.signed_uri` requires `use_account_sas` as the second parameter. -* Removed the support for the Shared Key Lite. - -Tracking Breaking Changes in 0.10.0-preview - -ALL -* Require "azure-storage" instead of "azure_storage". - -BLOB -* The `create_blob_pages` method is renamed to `put_blob_pages`. -* The `create_blob_block` method is renamed to `put_blob_block`. -* The `acquire_lease` method is renamed to `acquire_blob_lease`. -* The `renew_lease` method is renamed to `renew_blob_lease`. -* The `release_lease` method is renamed to `release_blob_lease`. -* The `break_lease` method is renamed to `break_blob_lease`. +* This module now only consists of functionalities to access Azure Storage Blob Service. +* Creating Blob Client using `Azure::Storage::Client.create` is now deprecated. To create a Blob client, users have to choose from `Azure::Storage::Blob::BlobService::create`, `Azure::Storage::Blob::BlobService::create_development`, ``Azure::Storage::Blob::BlobService::create_from_env`, `Azure::Storage::Blob::BlobService::create_from_connection_string` or `Azure::Storage::Blob::BlobService.new`. The parameters remain unchanged. +* The default `Content-Type` for a newly created page blob or append blob will now be `application/octet-stream`, which matches the description of [REST doc](https://docs.microsoft.com/en-us/rest/api/storageservices/put-blob) +* The API "Azure::Storage::Blob::BlobService::create_block_blob" can now upload content that is larger than 256MB instead of reporting failure. diff --git a/blob/ChangeLog.md b/blob/ChangeLog.md index 54232455..5ebe4e74 100644 --- a/blob/ChangeLog.md +++ b/blob/ChangeLog.md @@ -1,207 +1,12 @@ -2017.12 - version 1.0.0 +2018.1 - version 1.0.0 -BLOB -* Added following convenience APIs to support large payload upload from local to append or page blob. - - Azure::Storage::Blob::BlobService::create_page_blob_with_content +* This module now only consists of functionalities to access Azure Storage Blob Service. +* Creating Blob Client using `Azure::Storage::Client.create` is now deprecated. To create a Blob client, users have to choose from `Azure::Storage::Blob::BlobService::create`, `Azure::Storage::Blob::BlobService::create_development`, ``Azure::Storage::Blob::BlobService::create_from_env`, `Azure::Storage::Blob::BlobService::create_from_connection_string` or `Azure::Storage::Blob::BlobService.new`. The parameters remain unchanged. +* Added following convenience APIs to support large payload upload from given content to append or page blob, to make block blob consistent with API name, added an alias for `create_block_blob` as well. + - Azure::Storage::Blob::BlobService::create_block_blob_from_content + - Azure::Storage::Blob::BlobService::create_page_blob_from_content - Azure::Storage::Blob::BlobService::create_append_blob_from_content * Added the support for `Azure::Storage::Blob::BlobService::create_block_blob` to handle large payload that used to require making multiple `Azure::Storage::Blob::BlobService::put_blob_block` calls and calling `Azure::Storage::Blob::BlobService::commit_blob_blocks`. - -File -* Added convenience API `Azure::Storage::File::FileService::create_file_with_content` to support large payload upload from local to file. - -2017.11 - version 0.15.0-preview - -ALL -* Added the support for the location mode in the API options. -* Added the support for retrying according to the location mode. - -BLOB -* Added the support for retrieving statistics related to replication for the Blob service. -* Added the support for anonymous read access of public containers. -* Added full lease ID header support for following methods: - - Azure::Storage::Blob::BlobService::get_container_properties - - Azure::Storage::Blob::BlobService::get_container_metadata - - Azure::Storage::Blob::BlobService::get_container_acl - - Azure::Storage::Blob::BlobService::set_container_metadata - - Azure::Storage::Blob::BlobService::set_container_acl - - Azure::Storage::Blob::BlobService::delete_container - - Azure::Storage::Blob::BlobService::create_block_blob - - Azure::Storage::Blob::BlobService::create_page_blob - - Azure::Storage::Blob::BlobService::create_append_blob - - Azure::Storage::Blob::BlobService::get_blob - - Azure::Storage::Blob::BlobService::get_blob_properties - - Azure::Storage::Blob::BlobService::set_blob_properties - - Azure::Storage::Blob::BlobService::get_blob_metadata - - Azure::Storage::Blob::BlobService::set_blob_metadata - - Azure::Storage::Blob::BlobService::create_blob_snapshot - - Azure::Storage::Blob::BlobService::copy_blob - - Azure::Storage::Blob::BlobService::copy_blob_from_uri - - Azure::Storage::Blob::BlobService::delete_blob - - Azure::Storage::Blob::BlobService::put_blob_block - - Azure::Storage::Blob::BlobService::commit_blob_blocks - - Azure::Storage::Blob::BlobService::list_blob_blocks - - Azure::Storage::Blob::BlobService::put_blob_pages - - Azure::Storage::Blob::BlobService::list_page_blob_ranges - - Azure::Storage::Blob::BlobService::incremental_copy_blob - - Azure::Storage::Blob::BlobService::append_blob_block - -Queue -* Added the support for retrieving statistics related to replication for the Queue service. - -Table -* Added the support for retrieving statistics related to replication for the Table service. - -2017.09 - version 0.14.0-preview - -ALL -* Added configuration file for Rubocop and auto-resolved coding style issue. - -BLOB -* The `Azure::Storage::Blob::list_page_blob_ranges` API now accepts `:previous_snapshot` as an optional parameter, that specifies that the response returns pages that have been updated or cleared since the snapshot specified by `:previous_snapshot` was taken. -* The `Azure::Storage::Blob::Blob` object now has an attribute `:encrypted` showing if the blob or blob related request has been encrypted. -* The `Azure::Storage::Blob::BlobService::list_containers` and `Azure::Storage::Blob::BlobService::get_container_properties` will now also return public access level for each container. -* The stored Content-MD5 property is now returned when requesting a range of a blob. Previously this was only returned for full blob downloads. `Azure::Storage::Blob::Blob.properties[:content_md5]` will always hold the stored Content_MD5 property, and `Azure::Storage::Blob::Blob.properties[:range_md5]` will always represent the MD5 for the content returned from the server. -* Added an API `Azure::Storage::Blob::BlobService::incremental_copy_blob` to support [incremental copy](https://docs.microsoft.com/en-us/rest/api/storageservices/incremental-copy-blob) for page blob snapshots. - -FILE -* The stored Content-MD5 property is now returned when requesting a range of a file. Previously this was only returned for full file downloads. `Azure::Storage::File::File.properties[:content_md5]` will always hold the stored Content_MD5 property, and `Azure::Storage::File::File.properties[:range_md5]` will always represent the MD5 for the content returned from the server. - -QUEUE -* The return type of `Azure::Storage::Queue::create_message` is changed from `nil` to an `Azure::Storage::Queue::Message` object. - -FILE -* The API `Azure::Storage::File::list_directories_and_files` now also accepts `:prefix` as an optional parameter. The return value will be filtered with the specified prefix if set. - -2017.09 - version 0.13.0-preview - -ALL -* Removed Nokogiri from Gemfile because it causes bundler fail to install azure-storage. Added it back to runtime dependency and explicitly require user to install the correct version of Nokogiri in README.md. -* Service version is upgraded to 2016-05-31. - -BLOB -* Block size can now be up to 100MB. - -TABLE -* The return type `Azure::Service::EnumerationResult` of `query_tables` has a changed structure. Now the `'updated'` will not be contained, and is flattened to a structure in the form of `{ {"TableName" => "tableone"}, {"TableName" => "tabletwo"}, {"TableName" => "tablethree"}}`. -* The `Azure::Storage::Table::Entity` does not contain `:table` and `updated` anymore. The updated time can be found in `:properties`. -* The return type of `get_table` is changed to a Hash that contains full metadata returned from the server when query the table. -* The method `Azure::Storage::Table::EdmType::unserialize_query_value` is renamed to `deserialize_value`. - -2017.08 - version 0.12.3-preview - -ALL -* Added Nokogiri as a gem into Gemfile, resolving an issue where bundler failed to recognize that the dependency exists after installation. - -2017.08 - version 0.12.2-preview - -ALL -* Removed Nokogiri as a dependency to resolve conflict version caused by azure-core also depending on Nokogiri. - -2017.04 - version 0.12.1-preview - -ALL -* Relaxed constraint on Nokogiri version dependency to allow Nokogiri 1.7.x for Ruby 2.1 and later, but preserving support for Ruby 1.9.3. (Note that this may require updating to Bundler 1.13 or later if you're using Ruby 1.9 or 2.0.) - -2017.02 - version 0.12.0-preview - -ALL -* Fixed the issue where `should_retry?` in the retry_filter.rb overwrites the result from derived `apply_retry_policy`. [#76](https://github.com/Azure/azure-storage-ruby/issues/76) -* Fixed the issue where `Azure::Storage::Client.create_from_connection_string` throws an exception. [#77](https://github.com/Azure/azure-storage-ruby/issues/77) -* Added the support for setting the "timeout" option in `get_service_properties` and `set_service_properties`. - -BLOB -* Added the metadata to the returning instance when creates a blob. -* Added `transactional_md5` to the options of `put_blob_pages`. - -FILE -* Added File Service support, targeting storage service version 2015-04-05. - -2016.12 - version 0.11.5-preview - -ALL -* Added the support for setting customer user agent. [#71](https://github.com/Azure/azure-storage-ruby/issues/71) -* Added the support for hooking in sending requests. - -2016.11 - version 0.11.4-preview - -ALL -* Removed the unnecessary dependencies. [#55](https://github.com/Azure/azure-storage-ruby/issues/55), [#67](https://github.com/Azure/azure-storage-ruby/issues/67) - -BLOB -* Fixed the issue when checking the content encoding. -* Fixed the wrong "Content-Encoding" value in the test cases. -* Fixed the issue where it cannot use the `create_block_blob` method with an IO/File object. [#61](https://github.com/Azure/azure-storage-ruby/issues/61) - -2016.10 - version 0.11.3-preview - -ALL -* Fixed an issue in retry policies. - -2016.10 - version 0.11.2-preview - -ALL -* Fixed the issue where it retries on HTTP 4xx errors. - -BLOB -* Fixed the issue of wrong "Content-Encoding". [#49](https://github.com/Azure/azure-storage-ruby/issues/49) - -2016.09 - version 0.11.1-preview - -ALL -* Added the support for setting the client request ID via the "request_id" parameter. -* Added the retry for the timeout errors. -* Added the retry for the connection reset error. - -BLOB -* Fixed the issue where "list_blobs" doesn't work when delimiter is specified. [#41](https://github.com/Azure/azure-storage-ruby/issues/41) - -2016.08 - version 0.11.0-preview - -ALL -* Added the support for the account shared access signature. -* Removed the support for the Shared Key Lite. - -BLOB -* Added the support for the "add" and "create" permissions in the blob service shared access signature. - -FILE -* Added the support for the "create" permission in the file service shared access signature. - -2016.06 - version 0.10.2-preview - -ALL -* Fixed the issue that cannot run against storage emulator on Windows. -* Fixed the issue that it doesn't run as a singleton when it calls Azure::Storage.setup. -* Updated to storage service version 2015-04-05. - -2016.05 - version 0.10.1-preview - -ALL -* Replaced the core module by the 'azure-core' gem. -* Stopped maintaining the support for Ruby 1.9. -* Added support for retry filters. - -2016.03 - version 0.10.0-preview - -ALL -* Separated out parts of Azure Storage previously found in the Azure SDK 0.7.0 to establish an independent release cycle. -* Supported Ruby 1.9.3, 2.0, 2.1 and 2.2. -* Updated to storage service version 2015-02-21. -* Fixed issue where previous query's parameters were used, causing authentication to fail (https://github.com/Azure/azure-sdk-for-ruby/issues/276). -* Fixed the issue that the Content-MD5 is overwritten when request body exists, regardless the input value. -* Refined the code for setting service properties to be compatible with the XML schema. - -BLOB -* Added support for container lease operations. -* Added support for changing the ID of an existing lease. -* Added support for copying from source blob URI. -* Added support for aborting copying a blob. -* Added support for creating an append blob. -* Added support for appending a block to an appendblob. -* Added support for the content disposition property of a blob. -* Added support for resizing a page blob. -* Added support for setting a page blob's sequence number. -* Fixed the issue where conditional headers for some APIs could not be set. -* Fixed the issue where the request fails when calling list_page_blob_ranges with start range and end range. -* Renamed create_blob_block to put_blob_block. -* Renamed create_blob_pages to put_blob_pages. +* The default `Content-Type` for a newly created page blob or append blob will now be `application/octet-stream`, which matches the description of [REST doc](https://docs.microsoft.com/en-us/rest/api/storageservices/put-blob) +* Resolved an issue where parsing XML content would sometimes throw unexpected failures. +* Resolved an issue where users send "increment" as `:sequence_number_action` option for `Azure::Storage::Blob::BlobService::set_blob_properties` would not be recognized. diff --git a/blob/README.md b/blob/README.md index 4b743388..b9de4f0d 100644 --- a/blob/README.md +++ b/blob/README.md @@ -1,42 +1,34 @@ # Microsoft Azure Storage Blob Client Library for Ruby -[![Gem Version](https://badge.fury.io/rb/azure-storage.svg)](https://badge.fury.io/rb/azure-storage) +[![Gem Version](https://badge.fury.io/rb/azure-storage-blob.svg)](https://badge.fury.io/rb/azure-storage-blob) * Master: [![Master Build Status](https://travis-ci.org/Azure/azure-storage-ruby.svg?branch=master)](https://travis-ci.org/Azure/azure-storage-ruby/branches) [![Coverage Status](https://coveralls.io/repos/github/Azure/azure-storage-ruby/badge.svg?branch=master)](https://coveralls.io/github/Azure/azure-storage-ruby?branch=master) * Dev: [![Dev Build Status](https://travis-ci.org/Azure/azure-storage-ruby.svg?branch=dev)](https://travis-ci.org/Azure/azure-storage-ruby/branches) [![Coverage Status](https://coveralls.io/repos/github/Azure/azure-storage-ruby/badge.svg?branch=dev)](https://coveralls.io/github/Azure/azure-storage-ruby?branch=dev) -This project provides a Ruby package that makes it easy to access and manage Microsoft Azure Storage Services. +This project provides a Ruby package that makes it easy to access and manage Microsoft Azure Storage Blob Services. -# Library Features - -* [Blobs](#blobs) -* [Tables](#tables) -* [Queues](#queues) -* [Files](#files) # Supported Ruby Versions -* Ruby 2.0 -* Ruby 2.1 -* Ruby 2.2 +* Ruby 1.9.3 to 2.5 Note: * x64 Ruby for Windows is known to have some compatibility issues. -* azure-storage depends on gem nokogiri. For Ruby version lower than 2.2, please install the compatible nokogiri before trying to install azure-storage. +* azure-storage-blob depends on gem nokogiri. For Ruby version lower than 2.2, please install the compatible nokogiri before trying to install azure-storage-blob. # Getting Started ## Install the rubygem package -You can install the azure rubygem package directly. +You can install the azure storage blob rubygem package directly. ```bash -gem install azure-storage --pre +gem install azure-storage-blob ``` ## Setup Connection -You can use this SDK against the Microsoft Azure Storage Services in the cloud, or against the local Storage Emulator if you are on Windows. +You can use this Client Library against the Microsoft Azure Storage Blob Services in the cloud, or against the local Storage Emulator if you are on Windows. There are two ways you can set up the connections: @@ -49,17 +41,18 @@ There are two ways you can set up the connections: ```ruby - require 'azure/storage' + require 'azure/storage/blob' - # Setup a specific instance of an Azure::Storage::Client - client = Azure::Storage::Client.create(:storage_account_name => 'your account name', :storage_access_key => 'your access key') + # Setup a specific instance of an Azure::Storage::Blob::BlobService + blob_client = Azure::Storage::Blob::BlobService.create(storage_account_name: 'your account name', storage_access_key: 'your access key') - # Or create a client and store as a singleton - Azure::Storage.setup(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - # Then you can either call Azure::Storage.client.some_method or Azure::Storage.some_method to invoke a method on the Storage Client + # Or create a client and initialize with it. + require 'azure/storage/common' + common_client = Azure::Storage::Common::Client.create(storage_account_name: 'your account name', storage_access_key: 'your access key') + blob_client = Azure::Storage::Blob::BlobService.new(client: common_client) # Configure a ca_cert.pem file if you are having issues with ssl peer verification - client.ca_file = './ca_file.pem' + blob_client.ca_file = './ca_file.pem' ``` @@ -67,18 +60,18 @@ There are two ways you can set up the connections: ```ruby - require 'azure/storage' - client = Azure::Storage::Client.create_development + require 'azure/storage/blob' + client = Azure::Storage::Blob::BlobService.create_development # Or create by options and provide your own proxy_uri - client = Azure::Storage::Client.create(:use_development_storage => true, :development_storage_proxy_uri => 'your proxy uri') + client = Azure::Storage::Blob::BlobService.create(:use_development_storage => true, :development_storage_proxy_uri => 'your proxy uri') ``` ### Via Environment Variables -* Against Microsoft Azure Storage Services in the cloud +* Against Microsoft Azure Storage Blob Services in the cloud ```bash export AZURE_STORAGE_ACCOUNT = @@ -99,230 +92,58 @@ There are two ways you can set up the connections: # Usage - -## Blobs + +## Usage ```ruby -# Require the azure storage rubygem -require 'azure/storage' +# Require the azure storage blob rubygem +require 'azure/storage/blob' -# Setup a specific instance of an Azure::Storage::Client -client = Azure::Storage::Client.create(:storage_account_name => 'your account name', :storage_access_key => 'your access key') +# Setup a specific instance of an Azure::Storage::Blob::BlobService +client = Azure::Storage::Blob::BlobService.create(:storage_account_name => 'your account name', :storage_access_key => 'your access key') # Alternatively, create a client that can anonymously access public containers for read operations -client = Azure::Storage::Client.create(storage_blob_host: "https://youraccountname.blob.core.windows.net") - -# Get an azure storage blob service object from a specific instance of an Azure::Storage::Client -blobs = client.blob_client - -# Or setup the client as a singleton -Azure::Storage.setup(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Create an azure storage blob service object after you set up the credentials -blobs = Azure::Storage::Blob::BlobService.new +client = Azure::Storage::Blob::BlobService.create(storage_blob_host: "https://youraccountname.blob.core.windows.net") # Add retry filter to the service object -blobs.with_filter(Azure::Storage::Core::Filter::ExponentialRetryPolicyFilter.new) +require "azure/storage/common" +client.with_filter(Azure::Storage::Common::Core::Filter::ExponentialRetryPolicyFilter.new) # Create a container -container = blobs.create_container('test-container') +container = client.create_container('test-container') # Upload a Blob content = ::File.open('test.jpg', 'rb') { |file| file.read } -blobs.create_block_blob(container.name, 'image-blob', content) +client.create_block_blob(container.name, 'image-blob', content) # List containers -blobs.list_containers() +client.list_containers() # List Blobs -blobs.list_blobs(container.name) +client.list_blobs(container.name) # Download a Blob -blob, content = blobs.get_blob(container.name, 'image-blob') +blob, content = client.get_blob(container.name, 'image-blob') ::File.open('download.png', 'wb') {|f| f.write(content)} # Delete a Blob -blobs.delete_blob(container.name, 'image-blob') - -``` - -## Tables - -```ruby - -# Require the azure storage rubygem -require 'azure/storage' - -# Setup a specific instance of an Azure::Storage::Client -client = Azure::Storage::Client.create(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Get an azure storage table service object from a specific instance of an Azure::Storage::Client -tables = client.table_client - -# Or setup the client as a singleton -Azure::Storage.setup(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Create an azure storage table service object after you set up the credentials -tables = Azure::Storage::Table::TableService.new - -# Add retry filter to the service object -tables.with_filter(Azure::Storage::Core::Filter::ExponentialRetryPolicyFilter.new) - -# Create a table -tables.create_table('testtable') - -# Insert an entity -entity = { content: 'test entity', PartitionKey: 'test-partition-key', RowKey: '1' } -tables.insert_entity('testtable', entity) - -# Get an entity -result = tables.get_entity('testtable', 'test-partition-key', '1') - -# Update an entity -result.properties['content'] = 'test entity with updated content' -tables.update_entity(result.table, result.properties) - -# Query entities -query = { :filter => "content eq 'test entity'" } -result, token = tables.query_entities('testtable', query) - -# Delete an entity -tables.delete_entity('testtable', 'test-partition-key', '1') - -# delete a table -tables.delete_table('testtable') - -``` - - -## Queues - -```ruby - -# Require the azure storage rubygem -require 'azure/storage' - -# Setup a specific instance of an Azure::Storage::Client -client = Azure::Storage::Client.create(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Get an azure storage queue service object from a specific instance of an Azure::Storage::Client -queues = client.queue_client - -# Or setup the client as a singleton -Azure::Storage.setup(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Create an azure storage queue service object after you set up the credentials -queues = Azure::Storage::Queue::QueueService.new - -# Add retry filter to the service object -queues.with_filter(Azure::Storage::Core::Filter::ExponentialRetryPolicyFilter.new) - -# Create a queue -queues.create_queue('test-queue') - -# Create a message -queues.create_message('test-queue', 'test message') - -# Get one or more messages with setting the visibility timeout -result = queues.list_messages('test-queue', 30, { number_of_messages: 10 }) - -# Get one or more messages without setting the visibility timeout -result = queues.peek_messages('test-queue', { number_of_messages: 10 }) - -# Update a message -message = queues.list_messages('test-queue', 30) -pop_receipt, time_next_visible = queues.update_message('test-queue', message[0].id, message[0].pop_receipt, 'updated test message', 30) - -# Delete a message -message = queues.list_messages('test-queue', 30) -queues.delete_message('test-queue', message[0].id, message[0].pop_receipt) - -# Delete a queue -queues.delete_queue('test-queue') +client.delete_blob(container.name, 'image-blob') ``` - -## Files - -```ruby -# Require the azure storage rubygem -require 'azure/storage' - -# Setup a specific instance of an Azure::Storage::Client -client = Azure::Storage::Client.create(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Get an azure storage file service object from a specific instance of an Azure::Storage::Client -files = client.file_client - -# Or setup the client as a singleton -Azure::Storage.setup(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Create an azure storage file service object after you set up the credentials -files = Azure::Storage::File::FileService.new - -# Add retry filter to the service object -files.with_filter(Azure::Storage::Core::Filter::ExponentialRetryPolicyFilter.new) - -# Create a share -share = files.create_share('test-share') - -# Create a directory -directory = files.create_directory(share.name, 'test-directory') - -# Create a file and update the file content -content = ::File.open('test.jpg', 'rb') { |file| file.read } -file = files.create_file(share.name, directory.name, 'test-file', content.size) -files.put_file_range(share.name, directory.name, file.name, 0, content.size - 1, content) - -# List shares -files.list_shares() - -# List directories and files -files.list_directories_and_files(share.name, directory.name) - -# Download a File -file, content = files.get_file(share.name, directory.name, file.name) -::File.open('download.png', 'wb') {|f| f.write(content)} - -# Delete a File -files.delete_file(share.name, directory.name, file.name) -``` - ## Customize the user-agent You can customize the user-agent string by setting your user agent prefix when creating the service client. ```ruby -# Require the azure storage rubygem -require "azure/storage" +# Require the azure storage blob rubygem +require "azure/storage/blob" # Setup a specific instance of an Azure::Storage::Client with :user_agent_prefix option -client = Azure::Storage::Client.create(:storage_account_name => "your account name", :storage_access_key => "your access key", :user_agent_prefix => "your application name") +client = Azure::Storage::Blob::BlobService.create(:storage_account_name => "your account name", :storage_access_key => "your access key", :user_agent_prefix => "your application name") ``` -# Getting Started for Contributors - -If you would like to become an active contributor to this project please follow the instructions provided in [Azure Projects Contribution Guidelines](http://azure.github.io/guidelines/). -You can find more details for contributing in the [CONTRIBUTING.md](CONTRIBUTING.md). - -# Provide Feedback - -If you encounter any bugs with the library please file an issue in the [Issues](https://github.com/Azure/azure-storage-ruby/issues) section of the project. - -# Azure Storage SDKs and Tooling - -* [Azure Storage Client Library for .Net](http://github.com/azure/azure-storage-net) -* [Azure Storage Client Library for Java](http://github.com/azure/azure-storage-java) -* [Azure Storage Client Library for Node.js](http://github.com/azure/azure-storage-node) -* [Azure Storage Client Library for Python](http://github.com/azure/azure-storage-python) -* [Azure Storage Client Library for Ruby](http://github.com/azure/azure-storage-ruby) -* [Azure Storage Client Library for C++](http://github.com/azure/azure-storage-cpp) -* [Azure Storage Client Library for iOS](http://github.com/azure/azure-storage-ios) -* [Azure Storage Client Library for Android](http://github.com/azure/azure-storage-android) -* [Azure Storage Data Movement Library](https://github.com/Azure/azure-storage-net-data-movement) - # Code of Conduct This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments. diff --git a/blob/lib/azure/storage/blob/blob_service.rb b/blob/lib/azure/storage/blob/blob_service.rb index 44c4b2be..a81a5879 100755 --- a/blob/lib/azure/storage/blob/blob_service.rb +++ b/blob/lib/azure/storage/blob/blob_service.rb @@ -138,12 +138,14 @@ def create_from_connection_string(connection_string, &block) # * +:use_path_style_uri+ - String. Whether use path style URI for specified endpoints # * +:ca_file+ - String. File path of the CA file if having issue with SSL # * +:user_agent_prefix+ - String. The user agent prefix that can identify the application calls the library + # * +:client+ - Azure::Storage::Common::Client. The common client used to initalize the service. # # The valid set of options include: # * Storage Emulator: +:use_development_storage+ required, +:development_storage_proxy_uri+ optionally # * Storage account name and key: +:storage_account_name+ and +:storage_access_key+ required, set +:storage_dns_suffix+ necessarily # * Storage account name and SAS token: +:storage_account_name+ and +:storage_sas_token+ required, set +:storage_dns_suffix+ necessarily # * Specified hosts and SAS token: At least one of the service host and SAS token. It's up to user to ensure the SAS token is suitable for the serivce + # * Azure::Storage::Common::Client: The common client used to initalize the service. This client can be initalized and used repeatedly. # * Anonymous Blob: only +:storage_blob_host+, if it is to only access blobs within a container # # Additional notes: diff --git a/blob/lib/azure/storage/blob/page.rb b/blob/lib/azure/storage/blob/page.rb index e7c342de..0a1f3a40 100644 --- a/blob/lib/azure/storage/blob/page.rb +++ b/blob/lib/azure/storage/blob/page.rb @@ -543,7 +543,7 @@ def set_sequence_number(container, blob, action, number, options = {}) # See http://msdn.microsoft.com/en-us/library/azure/dd179451.aspx # # Returns a Blob - def create_page_blob_with_content(container, blob, length, content, options = {}) + def create_page_blob_from_content(container, blob, length, content, options = {}) options[:content_type] = get_or_apply_content_type(content, options[:content_type]) create_page_blob(container, blob, length, options) diff --git a/common/BreakingChanges.md b/common/BreakingChanges.md index 94b645f9..98b81484 100644 --- a/common/BreakingChanges.md +++ b/common/BreakingChanges.md @@ -1,32 +1,4 @@ -Tracking Breaking Changes in 0.14.0-preview +Tracking Breaking Changes in 1.0.0 -QUEUE -* The return type of `Azure::Storage::Queue::create_message` is changed from `nil` to an `Azure::Storage::Queue::Message` object. - -Tracking Breaking Changes in 0.13.0-preview - -TABLE -* The return type `Azure::Service::EnumerationResult` of `query_tables` has a changed structure. Now the `'updated'` will not be contained, and is flattened to a structure in the form of `{ {"TableName" => "tableone"}, {"TableName" => "tabletwo"}, {"TableName" => "tablethree"}}`. -* The `Azure::Storage::Table::Entity` does not contain `:table` and `updated` anymore. The updated time can be found in `:properties`. -* The return type of `get_table` is changed to a Hash that contains full metadata returned from the server when query the table. -* The method `Azure::Storage::Table::EdmType::unserialize_query_value` is renamed to `deserialize_value`. - -Tracking Breaking Changes in 0.11.0-preview - -ALL -* `Azure::Storage::Core::Auth::SharedAccessSignature.generate` is renamed to `generate_service_sas_token`. -* `Azure::Storage::Core::Auth::SharedAccessSignature.signed_uri` requires `use_account_sas` as the second parameter. -* Removed the support for the Shared Key Lite. - -Tracking Breaking Changes in 0.10.0-preview - -ALL -* Require "azure-storage" instead of "azure_storage". - -BLOB -* The `create_blob_pages` method is renamed to `put_blob_pages`. -* The `create_blob_block` method is renamed to `put_blob_block`. -* The `acquire_lease` method is renamed to `acquire_blob_lease`. -* The `renew_lease` method is renamed to `renew_blob_lease`. -* The `release_lease` method is renamed to `release_blob_lease`. -* The `break_lease` method is renamed to `break_blob_lease`. +* This module now consists of functionalities to support service client library modules. +* All namespaces in this module now begin with "Azure::Storage::Common" instead of "Azure::Storage". diff --git a/common/CONTRIBUTING.md b/common/CONTRIBUTING.md deleted file mode 100644 index 9db89891..00000000 --- a/common/CONTRIBUTING.md +++ /dev/null @@ -1,74 +0,0 @@ -If you intend to contribute to the project, please make sure you've followed the instructions provided in the [Azure Projects Contribution Guidelines](http://azure.github.io/guidelines/). -## Project Setup -The Azure Storage development team uses Visual Studio Code so instructions will be tailored to that preference. However, any preferred IDE or other toolset should be usable. - -### Install -* Ruby 1.9.3, 2.0, 2.1 or 2.2. -* [Visual Studio Code](https://code.visualstudio.com/) - -### Development Environment Setup -To get the source code of the SDK via **git** just type: - -```bash -git clone https://github.com/Azure/azure-storage-ruby.git -cd ./azure-storage-ruby -``` - -Then, install Bundler and run bundler to install all the gem dependencies: - -```bash -gem install bundler -bundle install -``` - -## Tests - -### Configuration -If you would like to run the integration test suite, you will need to setup environment variables which will be used -during the integration tests. These tests will use these credentials to run live tests against Azure with the provided -credentials (you will be charged for usage, so verify the clean up scripts did their job at the end of a test run). - -The root of the project contains a [.env_sample](.env_sample) file. This dot file is a sample of the actual environment vars needed to -run the integration tests. - -Unit tests don't require real credentials and don't require to provide the .env file. - -Do the following to prepare your environment for integration tests: - -* Copy [.env_sample](.env_sample) to .env **relative to root of the project dir** -* Update .env with your credentials **.env is in the .gitignore, so should only reside locally** - -### Running -You can use the following commands to run: - -* All the tests: ``rake test``. **This will run integration tests if you have .env file or env vars setup** -* Run storage suite of tests: ``rake test:unit``, ``rake test:integration`` -* One particular test file: ``ruby -I"lib:test" ""`` - -### Testing Features -As you develop a feature, you'll need to write tests to ensure quality. Your changes should be covered by both unit tests and integration tests. You should also run existing tests related to your change to address any unexpected breaks. - -## Pull Requests - -### Guidelines -The following are the minimum requirements for any pull request that must be met before contributions can be accepted. -* Make sure you've signed the CLA before you start working on any change. -* Discuss any proposed contribution with the team via a GitHub issue **before** starting development. -* Code must be professional quality - * No style issues - * You should strive to mimic the style with which we have written the library - * Clean, well-commented, well-designed code - * Try to limit the number of commits for a feature to 1-2. If you end up having too many we may ask you to squash your changes into fewer commits. -* [ChangeLog.md](ChangeLog.md) needs to be updated describing the new change -* Thoroughly test your feature - -### Branching Policy -Changes should be based on the **dev** branch, not master as master is considered publicly released code. Each breaking change should be recorded in [BreakingChanges.md](BreakingChanges.md). - -### Adding Features for All Platforms -We strive to release each new feature for each of our environments at the same time. Therefore, we ask that all contributions be written for Ruby 1.9.3 and later. - -### Review Process -We expect all guidelines to be met before accepting a pull request. As such, we will work with you to address issues we find by leaving comments in your code. Please understand that it may take a few iterations before the code is accepted as we maintain high standards on code quality. Once we feel comfortable with a contribution, we will validate the change and accept the pull request. - -Thank you for any contributions! Please let the team know if you have any questions or concerns about our contribution policy. \ No newline at end of file diff --git a/common/ChangeLog.md b/common/ChangeLog.md index 54232455..3002ec48 100644 --- a/common/ChangeLog.md +++ b/common/ChangeLog.md @@ -1,207 +1,4 @@ -2017.12 - version 1.0.0 +2018.1 - version 1.0.0 -BLOB -* Added following convenience APIs to support large payload upload from local to append or page blob. - - Azure::Storage::Blob::BlobService::create_page_blob_with_content - - Azure::Storage::Blob::BlobService::create_append_blob_from_content -* Added the support for `Azure::Storage::Blob::BlobService::create_block_blob` to handle large payload that used to require making multiple `Azure::Storage::Blob::BlobService::put_blob_block` calls and calling `Azure::Storage::Blob::BlobService::commit_blob_blocks`. - -File -* Added convenience API `Azure::Storage::File::FileService::create_file_with_content` to support large payload upload from local to file. - -2017.11 - version 0.15.0-preview - -ALL -* Added the support for the location mode in the API options. -* Added the support for retrying according to the location mode. - -BLOB -* Added the support for retrieving statistics related to replication for the Blob service. -* Added the support for anonymous read access of public containers. -* Added full lease ID header support for following methods: - - Azure::Storage::Blob::BlobService::get_container_properties - - Azure::Storage::Blob::BlobService::get_container_metadata - - Azure::Storage::Blob::BlobService::get_container_acl - - Azure::Storage::Blob::BlobService::set_container_metadata - - Azure::Storage::Blob::BlobService::set_container_acl - - Azure::Storage::Blob::BlobService::delete_container - - Azure::Storage::Blob::BlobService::create_block_blob - - Azure::Storage::Blob::BlobService::create_page_blob - - Azure::Storage::Blob::BlobService::create_append_blob - - Azure::Storage::Blob::BlobService::get_blob - - Azure::Storage::Blob::BlobService::get_blob_properties - - Azure::Storage::Blob::BlobService::set_blob_properties - - Azure::Storage::Blob::BlobService::get_blob_metadata - - Azure::Storage::Blob::BlobService::set_blob_metadata - - Azure::Storage::Blob::BlobService::create_blob_snapshot - - Azure::Storage::Blob::BlobService::copy_blob - - Azure::Storage::Blob::BlobService::copy_blob_from_uri - - Azure::Storage::Blob::BlobService::delete_blob - - Azure::Storage::Blob::BlobService::put_blob_block - - Azure::Storage::Blob::BlobService::commit_blob_blocks - - Azure::Storage::Blob::BlobService::list_blob_blocks - - Azure::Storage::Blob::BlobService::put_blob_pages - - Azure::Storage::Blob::BlobService::list_page_blob_ranges - - Azure::Storage::Blob::BlobService::incremental_copy_blob - - Azure::Storage::Blob::BlobService::append_blob_block - -Queue -* Added the support for retrieving statistics related to replication for the Queue service. - -Table -* Added the support for retrieving statistics related to replication for the Table service. - -2017.09 - version 0.14.0-preview - -ALL -* Added configuration file for Rubocop and auto-resolved coding style issue. - -BLOB -* The `Azure::Storage::Blob::list_page_blob_ranges` API now accepts `:previous_snapshot` as an optional parameter, that specifies that the response returns pages that have been updated or cleared since the snapshot specified by `:previous_snapshot` was taken. -* The `Azure::Storage::Blob::Blob` object now has an attribute `:encrypted` showing if the blob or blob related request has been encrypted. -* The `Azure::Storage::Blob::BlobService::list_containers` and `Azure::Storage::Blob::BlobService::get_container_properties` will now also return public access level for each container. -* The stored Content-MD5 property is now returned when requesting a range of a blob. Previously this was only returned for full blob downloads. `Azure::Storage::Blob::Blob.properties[:content_md5]` will always hold the stored Content_MD5 property, and `Azure::Storage::Blob::Blob.properties[:range_md5]` will always represent the MD5 for the content returned from the server. -* Added an API `Azure::Storage::Blob::BlobService::incremental_copy_blob` to support [incremental copy](https://docs.microsoft.com/en-us/rest/api/storageservices/incremental-copy-blob) for page blob snapshots. - -FILE -* The stored Content-MD5 property is now returned when requesting a range of a file. Previously this was only returned for full file downloads. `Azure::Storage::File::File.properties[:content_md5]` will always hold the stored Content_MD5 property, and `Azure::Storage::File::File.properties[:range_md5]` will always represent the MD5 for the content returned from the server. - -QUEUE -* The return type of `Azure::Storage::Queue::create_message` is changed from `nil` to an `Azure::Storage::Queue::Message` object. - -FILE -* The API `Azure::Storage::File::list_directories_and_files` now also accepts `:prefix` as an optional parameter. The return value will be filtered with the specified prefix if set. - -2017.09 - version 0.13.0-preview - -ALL -* Removed Nokogiri from Gemfile because it causes bundler fail to install azure-storage. Added it back to runtime dependency and explicitly require user to install the correct version of Nokogiri in README.md. -* Service version is upgraded to 2016-05-31. - -BLOB -* Block size can now be up to 100MB. - -TABLE -* The return type `Azure::Service::EnumerationResult` of `query_tables` has a changed structure. Now the `'updated'` will not be contained, and is flattened to a structure in the form of `{ {"TableName" => "tableone"}, {"TableName" => "tabletwo"}, {"TableName" => "tablethree"}}`. -* The `Azure::Storage::Table::Entity` does not contain `:table` and `updated` anymore. The updated time can be found in `:properties`. -* The return type of `get_table` is changed to a Hash that contains full metadata returned from the server when query the table. -* The method `Azure::Storage::Table::EdmType::unserialize_query_value` is renamed to `deserialize_value`. - -2017.08 - version 0.12.3-preview - -ALL -* Added Nokogiri as a gem into Gemfile, resolving an issue where bundler failed to recognize that the dependency exists after installation. - -2017.08 - version 0.12.2-preview - -ALL -* Removed Nokogiri as a dependency to resolve conflict version caused by azure-core also depending on Nokogiri. - -2017.04 - version 0.12.1-preview - -ALL -* Relaxed constraint on Nokogiri version dependency to allow Nokogiri 1.7.x for Ruby 2.1 and later, but preserving support for Ruby 1.9.3. (Note that this may require updating to Bundler 1.13 or later if you're using Ruby 1.9 or 2.0.) - -2017.02 - version 0.12.0-preview - -ALL -* Fixed the issue where `should_retry?` in the retry_filter.rb overwrites the result from derived `apply_retry_policy`. [#76](https://github.com/Azure/azure-storage-ruby/issues/76) -* Fixed the issue where `Azure::Storage::Client.create_from_connection_string` throws an exception. [#77](https://github.com/Azure/azure-storage-ruby/issues/77) -* Added the support for setting the "timeout" option in `get_service_properties` and `set_service_properties`. - -BLOB -* Added the metadata to the returning instance when creates a blob. -* Added `transactional_md5` to the options of `put_blob_pages`. - -FILE -* Added File Service support, targeting storage service version 2015-04-05. - -2016.12 - version 0.11.5-preview - -ALL -* Added the support for setting customer user agent. [#71](https://github.com/Azure/azure-storage-ruby/issues/71) -* Added the support for hooking in sending requests. - -2016.11 - version 0.11.4-preview - -ALL -* Removed the unnecessary dependencies. [#55](https://github.com/Azure/azure-storage-ruby/issues/55), [#67](https://github.com/Azure/azure-storage-ruby/issues/67) - -BLOB -* Fixed the issue when checking the content encoding. -* Fixed the wrong "Content-Encoding" value in the test cases. -* Fixed the issue where it cannot use the `create_block_blob` method with an IO/File object. [#61](https://github.com/Azure/azure-storage-ruby/issues/61) - -2016.10 - version 0.11.3-preview - -ALL -* Fixed an issue in retry policies. - -2016.10 - version 0.11.2-preview - -ALL -* Fixed the issue where it retries on HTTP 4xx errors. - -BLOB -* Fixed the issue of wrong "Content-Encoding". [#49](https://github.com/Azure/azure-storage-ruby/issues/49) - -2016.09 - version 0.11.1-preview - -ALL -* Added the support for setting the client request ID via the "request_id" parameter. -* Added the retry for the timeout errors. -* Added the retry for the connection reset error. - -BLOB -* Fixed the issue where "list_blobs" doesn't work when delimiter is specified. [#41](https://github.com/Azure/azure-storage-ruby/issues/41) - -2016.08 - version 0.11.0-preview - -ALL -* Added the support for the account shared access signature. -* Removed the support for the Shared Key Lite. - -BLOB -* Added the support for the "add" and "create" permissions in the blob service shared access signature. - -FILE -* Added the support for the "create" permission in the file service shared access signature. - -2016.06 - version 0.10.2-preview - -ALL -* Fixed the issue that cannot run against storage emulator on Windows. -* Fixed the issue that it doesn't run as a singleton when it calls Azure::Storage.setup. -* Updated to storage service version 2015-04-05. - -2016.05 - version 0.10.1-preview - -ALL -* Replaced the core module by the 'azure-core' gem. -* Stopped maintaining the support for Ruby 1.9. -* Added support for retry filters. - -2016.03 - version 0.10.0-preview - -ALL -* Separated out parts of Azure Storage previously found in the Azure SDK 0.7.0 to establish an independent release cycle. -* Supported Ruby 1.9.3, 2.0, 2.1 and 2.2. -* Updated to storage service version 2015-02-21. -* Fixed issue where previous query's parameters were used, causing authentication to fail (https://github.com/Azure/azure-sdk-for-ruby/issues/276). -* Fixed the issue that the Content-MD5 is overwritten when request body exists, regardless the input value. -* Refined the code for setting service properties to be compatible with the XML schema. - -BLOB -* Added support for container lease operations. -* Added support for changing the ID of an existing lease. -* Added support for copying from source blob URI. -* Added support for aborting copying a blob. -* Added support for creating an append blob. -* Added support for appending a block to an appendblob. -* Added support for the content disposition property of a blob. -* Added support for resizing a page blob. -* Added support for setting a page blob's sequence number. -* Fixed the issue where conditional headers for some APIs could not be set. -* Fixed the issue where the request fails when calling list_page_blob_ranges with start range and end range. -* Renamed create_blob_block to put_blob_block. -* Renamed create_blob_pages to put_blob_pages. +* This module now consists of functionalities to support service client library modules. +* All namespaces in this module now begin with "Azure::Storage::Common" instead of "Azure::Storage". diff --git a/common/README.md b/common/README.md index d56cbe34..243550a4 100644 --- a/common/README.md +++ b/common/README.md @@ -1,44 +1,35 @@ # Microsoft Azure Storage Common Client Library for Ruby -[![Gem Version](https://badge.fury.io/rb/azure-storage.svg)](https://badge.fury.io/rb/azure-storage) +[![Gem Version](https://badge.fury.io/rb/azure-storage-common.svg)](https://badge.fury.io/rb/azure-storage-common) * Master: [![Master Build Status](https://travis-ci.org/Azure/azure-storage-ruby.svg?branch=master)](https://travis-ci.org/Azure/azure-storage-ruby/branches) [![Coverage Status](https://coveralls.io/repos/github/Azure/azure-storage-ruby/badge.svg?branch=master)](https://coveralls.io/github/Azure/azure-storage-ruby?branch=master) * Dev: [![Dev Build Status](https://travis-ci.org/Azure/azure-storage-ruby.svg?branch=dev)](https://travis-ci.org/Azure/azure-storage-ruby/branches) [![Coverage Status](https://coveralls.io/repos/github/Azure/azure-storage-ruby/badge.svg?branch=dev)](https://coveralls.io/github/Azure/azure-storage-ruby?branch=dev) -This project provides a Ruby package that makes it easy to access and manage Microsoft Azure Storage Services. - -# Library Features - -* [Blobs](#blobs) -* [Tables](#tables) -* [Queues](#queues) -* [Files](#files) +This project provides a Ruby package that supports service client libraries. # Supported Ruby Versions -* Ruby 2.0 -* Ruby 2.1 -* Ruby 2.2 +* Ruby 1.9.3 to 2.5 Note: * x64 Ruby for Windows is known to have some compatibility issues. -* azure-storage depends on gem nokogiri. For Ruby version lower than 2.2, please install the compatible nokogiri before trying to install azure-storage. +* azure-storage-common depends on gem nokogiri. For Ruby version lower than 2.2, please install the compatible nokogiri before trying to install azure-storage. # Getting Started ## Install the rubygem package -You can install the azure rubygem package directly. +You can install the azure storage common rubygem package directly. ```bash -gem install azure-storage --pre +gem install azure-storage-common ``` -## Setup Connection +## Create client -You can use this SDK against the Microsoft Azure Storage Services in the cloud, or against the local Storage Emulator if you are on Windows. +You can use this module to create client that can be later shared by service modules, to avoid repeating code of creating storage client. -There are two ways you can set up the connections: +There are two ways you can create the client: 1. [via code](#via-code) 2. [via environment variables](#via-environment-variables) @@ -49,14 +40,10 @@ There are two ways you can set up the connections: ```ruby - require 'azure/storage' - - # Setup a specific instance of an Azure::Storage::Client - client = Azure::Storage::Client.create(:storage_account_name => 'your account name', :storage_access_key => 'your access key') + require 'azure/storage/common' - # Or create a client and store as a singleton - Azure::Storage.setup(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - # Then you can either call Azure::Storage.client.some_method or Azure::Storage.some_method to invoke a method on the Storage Client + # Setup a specific instance of an Azure::Storage::Common::Client + client = Azure::Storage::Common::Client.create(:storage_account_name => 'your account name', :storage_access_key => 'your access key') # Configure a ca_cert.pem file if you are having issues with ssl peer verification client.ca_file = './ca_file.pem' @@ -67,11 +54,11 @@ There are two ways you can set up the connections: ```ruby - require 'azure/storage' - client = Azure::Storage::Client.create_development + require 'azure/storage/common' + client = Azure::Storage::Common::Client.create_development # Or create by options and provide your own proxy_uri - client = Azure::Storage::Client.create(:use_development_storage => true, :development_storage_proxy_uri => 'your proxy uri') + client = Azure::Storage::Common::Client.create(:use_development_storage => true, :development_storage_proxy_uri => 'your proxy uri') ``` @@ -99,230 +86,22 @@ There are two ways you can set up the connections: # Usage - -## Blobs - -```ruby - -# Require the azure storage rubygem -require 'azure/storage' - -# Setup a specific instance of an Azure::Storage::Client -client = Azure::Storage::Client.create(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Alternatively, create a client that can anonymously access public containers for read operations -client = Azure::Storage::Client.create(storage_blob_host: "https://youraccountname.blob.core.windows.net") - -# Get an azure storage blob service object from a specific instance of an Azure::Storage::Client -blobs = client.blob_client - -# Or setup the client as a singleton -Azure::Storage.setup(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Create an azure storage blob service object after you set up the credentials -blobs = Azure::Storage::Blob::BlobService.new - -# Add retry filter to the service object -blobs.with_filter(Azure::Storage::Core::Filter::ExponentialRetryPolicyFilter.new) - -# Create a container -container = blobs.create_container('test-container') - -# Upload a Blob -content = ::File.open('test.jpg', 'rb') { |file| file.read } -blobs.create_block_blob(container.name, 'image-blob', content) - -# List containers -blobs.list_containers() - -# List Blobs -blobs.list_blobs(container.name) - -# Download a Blob -blob, content = blobs.get_blob(container.name, 'image-blob') -::File.open('download.png', 'wb') {|f| f.write(content)} - -# Delete a Blob -blobs.delete_blob(container.name, 'image-blob') - -``` - -## Tables - -```ruby - -# Require the azure storage rubygem -require 'azure/storage' - -# Setup a specific instance of an Azure::Storage::Client -client = Azure::Storage::Client.create(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Get an azure storage table service object from a specific instance of an Azure::Storage::Client -tables = client.table_client - -# Or setup the client as a singleton -Azure::Storage.setup(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Create an azure storage table service object after you set up the credentials -tables = Azure::Storage::Table::TableService.new - -# Add retry filter to the service object -tables.with_filter(Azure::Storage::Core::Filter::ExponentialRetryPolicyFilter.new) - -# Create a table -tables.create_table('testtable') - -# Insert an entity -entity = { content: 'test entity', PartitionKey: 'test-partition-key', RowKey: '1' } -tables.insert_entity('testtable', entity) - -# Get an entity -result = tables.get_entity('testtable', 'test-partition-key', '1') - -# Update an entity -result.properties['content'] = 'test entity with updated content' -tables.update_entity(result.table, result.properties) - -# Query entities -query = { :filter => "content eq 'test entity'" } -result, token = tables.query_entities('testtable', query) - -# Delete an entity -tables.delete_entity('testtable', 'test-partition-key', '1') - -# delete a table -tables.delete_table('testtable') - -``` - - -## Queues + +## Shared Access Signature generation ```ruby -# Require the azure storage rubygem -require 'azure/storage' +require "azure/storage/common" -# Setup a specific instance of an Azure::Storage::Client -client = Azure::Storage::Client.create(:storage_account_name => 'your account name', :storage_access_key => 'your access key') +# Creating an instance of `Azure::Storage::Common::Core::SharedAccessSignature` +generator = Azure::Storage::Common::Core::SharedAccessSignature.new(your_account_name, your_access_key) -# Get an azure storage queue service object from a specific instance of an Azure::Storage::Client -queues = client.queue_client +# The generator now can be used to create service SAS or account SAS. +generator.generate_service_sas_token(my_path_or_table_name, my_sas_options) +generator.generate_account_sas_token(my_account_sas_options) +# For details about the possible options, please reference the document of the class `Azure::Storage::Common::Core::SharedAccessSignature` -# Or setup the client as a singleton -Azure::Storage.setup(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Create an azure storage queue service object after you set up the credentials -queues = Azure::Storage::Queue::QueueService.new - -# Add retry filter to the service object -queues.with_filter(Azure::Storage::Core::Filter::ExponentialRetryPolicyFilter.new) - -# Create a queue -queues.create_queue('test-queue') - -# Create a message -queues.create_message('test-queue', 'test message') - -# Get one or more messages with setting the visibility timeout -result = queues.list_messages('test-queue', 30, { number_of_messages: 10 }) - -# Get one or more messages without setting the visibility timeout -result = queues.peek_messages('test-queue', { number_of_messages: 10 }) - -# Update a message -message = queues.list_messages('test-queue', 30) -pop_receipt, time_next_visible = queues.update_message('test-queue', message[0].id, message[0].pop_receipt, 'updated test message', 30) - -# Delete a message -message = queues.list_messages('test-queue', 30) -queues.delete_message('test-queue', message[0].id, message[0].pop_receipt) - -# Delete a queue -queues.delete_queue('test-queue') - -``` - - -## Files - -```ruby -# Require the azure storage rubygem -require 'azure/storage' - -# Setup a specific instance of an Azure::Storage::Client -client = Azure::Storage::Client.create(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Get an azure storage file service object from a specific instance of an Azure::Storage::Client -files = client.file_client - -# Or setup the client as a singleton -Azure::Storage.setup(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Create an azure storage file service object after you set up the credentials -files = Azure::Storage::File::FileService.new - -# Add retry filter to the service object -files.with_filter(Azure::Storage::Core::Filter::ExponentialRetryPolicyFilter.new) - -# Create a share -share = files.create_share('test-share') - -# Create a directory -directory = files.create_directory(share.name, 'test-directory') - -# Create a file and update the file content -content = ::File.open('test.jpg', 'rb') { |file| file.read } -file = files.create_file(share.name, directory.name, 'test-file', content.size) -files.put_file_range(share.name, directory.name, file.name, 0, content.size - 1, content) - -# List shares -files.list_shares() - -# List directories and files -files.list_directories_and_files(share.name, directory.name) - -# Download a File -file, content = files.get_file(share.name, directory.name, file.name) -::File.open('download.png', 'wb') {|f| f.write(content)} - -# Delete a File -files.delete_file(share.name, directory.name, file.name) ``` - -## Customize the user-agent - -You can customize the user-agent string by setting your user agent prefix when creating the service client. - -```ruby -# Require the azure storage rubygem -require "azure/storage" - -# Setup a specific instance of an Azure::Storage::Client with :user_agent_prefix option -client = Azure::Storage::Client.create(:storage_account_name => "your account name", :storage_access_key => "your access key", :user_agent_prefix => "your application name") -``` - -# Getting Started for Contributors - -If you would like to become an active contributor to this project please follow the instructions provided in [Azure Projects Contribution Guidelines](http://azure.github.io/guidelines/). -You can find more details for contributing in the [CONTRIBUTING.md](CONTRIBUTING.md). - -# Provide Feedback - -If you encounter any bugs with the library please file an issue in the [Issues](https://github.com/Azure/azure-storage-ruby/issues) section of the project. - -# Azure Storage SDKs and Tooling - -* [Azure Storage Client Library for .Net](http://github.com/azure/azure-storage-net) -* [Azure Storage Client Library for Java](http://github.com/azure/azure-storage-java) -* [Azure Storage Client Library for Node.js](http://github.com/azure/azure-storage-node) -* [Azure Storage Client Library for Python](http://github.com/azure/azure-storage-python) -* [Azure Storage Client Library for Ruby](http://github.com/azure/azure-storage-ruby) -* [Azure Storage Client Library for C++](http://github.com/azure/azure-storage-cpp) -* [Azure Storage Client Library for iOS](http://github.com/azure/azure-storage-ios) -* [Azure Storage Client Library for Android](http://github.com/azure/azure-storage-android) -* [Azure Storage Data Movement Library](https://github.com/Azure/azure-storage-net-data-movement) - # Code of Conduct This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments. diff --git a/file/BreakingChanges.md b/file/BreakingChanges.md index 94b645f9..8f62e326 100644 --- a/file/BreakingChanges.md +++ b/file/BreakingChanges.md @@ -1,32 +1,4 @@ -Tracking Breaking Changes in 0.14.0-preview +Tracking Breaking Changes in 1.0.0 -QUEUE -* The return type of `Azure::Storage::Queue::create_message` is changed from `nil` to an `Azure::Storage::Queue::Message` object. - -Tracking Breaking Changes in 0.13.0-preview - -TABLE -* The return type `Azure::Service::EnumerationResult` of `query_tables` has a changed structure. Now the `'updated'` will not be contained, and is flattened to a structure in the form of `{ {"TableName" => "tableone"}, {"TableName" => "tabletwo"}, {"TableName" => "tablethree"}}`. -* The `Azure::Storage::Table::Entity` does not contain `:table` and `updated` anymore. The updated time can be found in `:properties`. -* The return type of `get_table` is changed to a Hash that contains full metadata returned from the server when query the table. -* The method `Azure::Storage::Table::EdmType::unserialize_query_value` is renamed to `deserialize_value`. - -Tracking Breaking Changes in 0.11.0-preview - -ALL -* `Azure::Storage::Core::Auth::SharedAccessSignature.generate` is renamed to `generate_service_sas_token`. -* `Azure::Storage::Core::Auth::SharedAccessSignature.signed_uri` requires `use_account_sas` as the second parameter. -* Removed the support for the Shared Key Lite. - -Tracking Breaking Changes in 0.10.0-preview - -ALL -* Require "azure-storage" instead of "azure_storage". - -BLOB -* The `create_blob_pages` method is renamed to `put_blob_pages`. -* The `create_blob_block` method is renamed to `put_blob_block`. -* The `acquire_lease` method is renamed to `acquire_blob_lease`. -* The `renew_lease` method is renamed to `renew_blob_lease`. -* The `release_lease` method is renamed to `release_blob_lease`. -* The `break_lease` method is renamed to `break_blob_lease`. +* This module now only consists of functionalities to access Azure Storage File Service. +* Creating File Client using `Azure::Storage::Client.create` is now deprecated. To create a File client, users have to choose from `Azure::Storage::File::FileService::create`, `Azure::Storage::File::FileService::create_development`, ``Azure::Storage::File::FileService::create_from_env`, `Azure::Storage::File::FileService::create_from_connection_string` or `Azure::Storage::File::FileService.new`. The parameters remain unchanged. diff --git a/file/CONTRIBUTING.md b/file/CONTRIBUTING.md deleted file mode 100644 index 9db89891..00000000 --- a/file/CONTRIBUTING.md +++ /dev/null @@ -1,74 +0,0 @@ -If you intend to contribute to the project, please make sure you've followed the instructions provided in the [Azure Projects Contribution Guidelines](http://azure.github.io/guidelines/). -## Project Setup -The Azure Storage development team uses Visual Studio Code so instructions will be tailored to that preference. However, any preferred IDE or other toolset should be usable. - -### Install -* Ruby 1.9.3, 2.0, 2.1 or 2.2. -* [Visual Studio Code](https://code.visualstudio.com/) - -### Development Environment Setup -To get the source code of the SDK via **git** just type: - -```bash -git clone https://github.com/Azure/azure-storage-ruby.git -cd ./azure-storage-ruby -``` - -Then, install Bundler and run bundler to install all the gem dependencies: - -```bash -gem install bundler -bundle install -``` - -## Tests - -### Configuration -If you would like to run the integration test suite, you will need to setup environment variables which will be used -during the integration tests. These tests will use these credentials to run live tests against Azure with the provided -credentials (you will be charged for usage, so verify the clean up scripts did their job at the end of a test run). - -The root of the project contains a [.env_sample](.env_sample) file. This dot file is a sample of the actual environment vars needed to -run the integration tests. - -Unit tests don't require real credentials and don't require to provide the .env file. - -Do the following to prepare your environment for integration tests: - -* Copy [.env_sample](.env_sample) to .env **relative to root of the project dir** -* Update .env with your credentials **.env is in the .gitignore, so should only reside locally** - -### Running -You can use the following commands to run: - -* All the tests: ``rake test``. **This will run integration tests if you have .env file or env vars setup** -* Run storage suite of tests: ``rake test:unit``, ``rake test:integration`` -* One particular test file: ``ruby -I"lib:test" ""`` - -### Testing Features -As you develop a feature, you'll need to write tests to ensure quality. Your changes should be covered by both unit tests and integration tests. You should also run existing tests related to your change to address any unexpected breaks. - -## Pull Requests - -### Guidelines -The following are the minimum requirements for any pull request that must be met before contributions can be accepted. -* Make sure you've signed the CLA before you start working on any change. -* Discuss any proposed contribution with the team via a GitHub issue **before** starting development. -* Code must be professional quality - * No style issues - * You should strive to mimic the style with which we have written the library - * Clean, well-commented, well-designed code - * Try to limit the number of commits for a feature to 1-2. If you end up having too many we may ask you to squash your changes into fewer commits. -* [ChangeLog.md](ChangeLog.md) needs to be updated describing the new change -* Thoroughly test your feature - -### Branching Policy -Changes should be based on the **dev** branch, not master as master is considered publicly released code. Each breaking change should be recorded in [BreakingChanges.md](BreakingChanges.md). - -### Adding Features for All Platforms -We strive to release each new feature for each of our environments at the same time. Therefore, we ask that all contributions be written for Ruby 1.9.3 and later. - -### Review Process -We expect all guidelines to be met before accepting a pull request. As such, we will work with you to address issues we find by leaving comments in your code. Please understand that it may take a few iterations before the code is accepted as we maintain high standards on code quality. Once we feel comfortable with a contribution, we will validate the change and accept the pull request. - -Thank you for any contributions! Please let the team know if you have any questions or concerns about our contribution policy. \ No newline at end of file diff --git a/file/ChangeLog.md b/file/ChangeLog.md index 54232455..9e08cc6a 100644 --- a/file/ChangeLog.md +++ b/file/ChangeLog.md @@ -1,207 +1,5 @@ -2017.12 - version 1.0.0 +2018.1 - version 1.0.0 -BLOB -* Added following convenience APIs to support large payload upload from local to append or page blob. - - Azure::Storage::Blob::BlobService::create_page_blob_with_content - - Azure::Storage::Blob::BlobService::create_append_blob_from_content -* Added the support for `Azure::Storage::Blob::BlobService::create_block_blob` to handle large payload that used to require making multiple `Azure::Storage::Blob::BlobService::put_blob_block` calls and calling `Azure::Storage::Blob::BlobService::commit_blob_blocks`. - -File -* Added convenience API `Azure::Storage::File::FileService::create_file_with_content` to support large payload upload from local to file. - -2017.11 - version 0.15.0-preview - -ALL -* Added the support for the location mode in the API options. -* Added the support for retrying according to the location mode. - -BLOB -* Added the support for retrieving statistics related to replication for the Blob service. -* Added the support for anonymous read access of public containers. -* Added full lease ID header support for following methods: - - Azure::Storage::Blob::BlobService::get_container_properties - - Azure::Storage::Blob::BlobService::get_container_metadata - - Azure::Storage::Blob::BlobService::get_container_acl - - Azure::Storage::Blob::BlobService::set_container_metadata - - Azure::Storage::Blob::BlobService::set_container_acl - - Azure::Storage::Blob::BlobService::delete_container - - Azure::Storage::Blob::BlobService::create_block_blob - - Azure::Storage::Blob::BlobService::create_page_blob - - Azure::Storage::Blob::BlobService::create_append_blob - - Azure::Storage::Blob::BlobService::get_blob - - Azure::Storage::Blob::BlobService::get_blob_properties - - Azure::Storage::Blob::BlobService::set_blob_properties - - Azure::Storage::Blob::BlobService::get_blob_metadata - - Azure::Storage::Blob::BlobService::set_blob_metadata - - Azure::Storage::Blob::BlobService::create_blob_snapshot - - Azure::Storage::Blob::BlobService::copy_blob - - Azure::Storage::Blob::BlobService::copy_blob_from_uri - - Azure::Storage::Blob::BlobService::delete_blob - - Azure::Storage::Blob::BlobService::put_blob_block - - Azure::Storage::Blob::BlobService::commit_blob_blocks - - Azure::Storage::Blob::BlobService::list_blob_blocks - - Azure::Storage::Blob::BlobService::put_blob_pages - - Azure::Storage::Blob::BlobService::list_page_blob_ranges - - Azure::Storage::Blob::BlobService::incremental_copy_blob - - Azure::Storage::Blob::BlobService::append_blob_block - -Queue -* Added the support for retrieving statistics related to replication for the Queue service. - -Table -* Added the support for retrieving statistics related to replication for the Table service. - -2017.09 - version 0.14.0-preview - -ALL -* Added configuration file for Rubocop and auto-resolved coding style issue. - -BLOB -* The `Azure::Storage::Blob::list_page_blob_ranges` API now accepts `:previous_snapshot` as an optional parameter, that specifies that the response returns pages that have been updated or cleared since the snapshot specified by `:previous_snapshot` was taken. -* The `Azure::Storage::Blob::Blob` object now has an attribute `:encrypted` showing if the blob or blob related request has been encrypted. -* The `Azure::Storage::Blob::BlobService::list_containers` and `Azure::Storage::Blob::BlobService::get_container_properties` will now also return public access level for each container. -* The stored Content-MD5 property is now returned when requesting a range of a blob. Previously this was only returned for full blob downloads. `Azure::Storage::Blob::Blob.properties[:content_md5]` will always hold the stored Content_MD5 property, and `Azure::Storage::Blob::Blob.properties[:range_md5]` will always represent the MD5 for the content returned from the server. -* Added an API `Azure::Storage::Blob::BlobService::incremental_copy_blob` to support [incremental copy](https://docs.microsoft.com/en-us/rest/api/storageservices/incremental-copy-blob) for page blob snapshots. - -FILE -* The stored Content-MD5 property is now returned when requesting a range of a file. Previously this was only returned for full file downloads. `Azure::Storage::File::File.properties[:content_md5]` will always hold the stored Content_MD5 property, and `Azure::Storage::File::File.properties[:range_md5]` will always represent the MD5 for the content returned from the server. - -QUEUE -* The return type of `Azure::Storage::Queue::create_message` is changed from `nil` to an `Azure::Storage::Queue::Message` object. - -FILE -* The API `Azure::Storage::File::list_directories_and_files` now also accepts `:prefix` as an optional parameter. The return value will be filtered with the specified prefix if set. - -2017.09 - version 0.13.0-preview - -ALL -* Removed Nokogiri from Gemfile because it causes bundler fail to install azure-storage. Added it back to runtime dependency and explicitly require user to install the correct version of Nokogiri in README.md. -* Service version is upgraded to 2016-05-31. - -BLOB -* Block size can now be up to 100MB. - -TABLE -* The return type `Azure::Service::EnumerationResult` of `query_tables` has a changed structure. Now the `'updated'` will not be contained, and is flattened to a structure in the form of `{ {"TableName" => "tableone"}, {"TableName" => "tabletwo"}, {"TableName" => "tablethree"}}`. -* The `Azure::Storage::Table::Entity` does not contain `:table` and `updated` anymore. The updated time can be found in `:properties`. -* The return type of `get_table` is changed to a Hash that contains full metadata returned from the server when query the table. -* The method `Azure::Storage::Table::EdmType::unserialize_query_value` is renamed to `deserialize_value`. - -2017.08 - version 0.12.3-preview - -ALL -* Added Nokogiri as a gem into Gemfile, resolving an issue where bundler failed to recognize that the dependency exists after installation. - -2017.08 - version 0.12.2-preview - -ALL -* Removed Nokogiri as a dependency to resolve conflict version caused by azure-core also depending on Nokogiri. - -2017.04 - version 0.12.1-preview - -ALL -* Relaxed constraint on Nokogiri version dependency to allow Nokogiri 1.7.x for Ruby 2.1 and later, but preserving support for Ruby 1.9.3. (Note that this may require updating to Bundler 1.13 or later if you're using Ruby 1.9 or 2.0.) - -2017.02 - version 0.12.0-preview - -ALL -* Fixed the issue where `should_retry?` in the retry_filter.rb overwrites the result from derived `apply_retry_policy`. [#76](https://github.com/Azure/azure-storage-ruby/issues/76) -* Fixed the issue where `Azure::Storage::Client.create_from_connection_string` throws an exception. [#77](https://github.com/Azure/azure-storage-ruby/issues/77) -* Added the support for setting the "timeout" option in `get_service_properties` and `set_service_properties`. - -BLOB -* Added the metadata to the returning instance when creates a blob. -* Added `transactional_md5` to the options of `put_blob_pages`. - -FILE -* Added File Service support, targeting storage service version 2015-04-05. - -2016.12 - version 0.11.5-preview - -ALL -* Added the support for setting customer user agent. [#71](https://github.com/Azure/azure-storage-ruby/issues/71) -* Added the support for hooking in sending requests. - -2016.11 - version 0.11.4-preview - -ALL -* Removed the unnecessary dependencies. [#55](https://github.com/Azure/azure-storage-ruby/issues/55), [#67](https://github.com/Azure/azure-storage-ruby/issues/67) - -BLOB -* Fixed the issue when checking the content encoding. -* Fixed the wrong "Content-Encoding" value in the test cases. -* Fixed the issue where it cannot use the `create_block_blob` method with an IO/File object. [#61](https://github.com/Azure/azure-storage-ruby/issues/61) - -2016.10 - version 0.11.3-preview - -ALL -* Fixed an issue in retry policies. - -2016.10 - version 0.11.2-preview - -ALL -* Fixed the issue where it retries on HTTP 4xx errors. - -BLOB -* Fixed the issue of wrong "Content-Encoding". [#49](https://github.com/Azure/azure-storage-ruby/issues/49) - -2016.09 - version 0.11.1-preview - -ALL -* Added the support for setting the client request ID via the "request_id" parameter. -* Added the retry for the timeout errors. -* Added the retry for the connection reset error. - -BLOB -* Fixed the issue where "list_blobs" doesn't work when delimiter is specified. [#41](https://github.com/Azure/azure-storage-ruby/issues/41) - -2016.08 - version 0.11.0-preview - -ALL -* Added the support for the account shared access signature. -* Removed the support for the Shared Key Lite. - -BLOB -* Added the support for the "add" and "create" permissions in the blob service shared access signature. - -FILE -* Added the support for the "create" permission in the file service shared access signature. - -2016.06 - version 0.10.2-preview - -ALL -* Fixed the issue that cannot run against storage emulator on Windows. -* Fixed the issue that it doesn't run as a singleton when it calls Azure::Storage.setup. -* Updated to storage service version 2015-04-05. - -2016.05 - version 0.10.1-preview - -ALL -* Replaced the core module by the 'azure-core' gem. -* Stopped maintaining the support for Ruby 1.9. -* Added support for retry filters. - -2016.03 - version 0.10.0-preview - -ALL -* Separated out parts of Azure Storage previously found in the Azure SDK 0.7.0 to establish an independent release cycle. -* Supported Ruby 1.9.3, 2.0, 2.1 and 2.2. -* Updated to storage service version 2015-02-21. -* Fixed issue where previous query's parameters were used, causing authentication to fail (https://github.com/Azure/azure-sdk-for-ruby/issues/276). -* Fixed the issue that the Content-MD5 is overwritten when request body exists, regardless the input value. -* Refined the code for setting service properties to be compatible with the XML schema. - -BLOB -* Added support for container lease operations. -* Added support for changing the ID of an existing lease. -* Added support for copying from source blob URI. -* Added support for aborting copying a blob. -* Added support for creating an append blob. -* Added support for appending a block to an appendblob. -* Added support for the content disposition property of a blob. -* Added support for resizing a page blob. -* Added support for setting a page blob's sequence number. -* Fixed the issue where conditional headers for some APIs could not be set. -* Fixed the issue where the request fails when calling list_page_blob_ranges with start range and end range. -* Renamed create_blob_block to put_blob_block. -* Renamed create_blob_pages to put_blob_pages. +* This module now only consists of functionalities to access Azure Storage File Service. +* Creating File Client using `Azure::Storage::Client.create` is now deprecated. To create a File client, users have to choose from `Azure::Storage::File::FileService::create`, `Azure::Storage::File::FileService::create_development`, ``Azure::Storage::File::FileService::create_from_env`, `Azure::Storage::File::FileService::create_from_connection_string` or `Azure::Storage::File::FileService.new`. The parameters remain unchanged. +* Added convenience API `Azure::Storage::File::FileService::create_file_from_content` to support large payload upload from local to file. diff --git a/file/README.md b/file/README.md index e36af015..8010beb4 100644 --- a/file/README.md +++ b/file/README.md @@ -1,42 +1,33 @@ # Microsoft Azure Storage File Client Library for Ruby -[![Gem Version](https://badge.fury.io/rb/azure-storage.svg)](https://badge.fury.io/rb/azure-storage) +[![Gem Version](https://badge.fury.io/rb/azure-storage-file.svg)](https://badge.fury.io/rb/azure-storage-file) * Master: [![Master Build Status](https://travis-ci.org/Azure/azure-storage-ruby.svg?branch=master)](https://travis-ci.org/Azure/azure-storage-ruby/branches) [![Coverage Status](https://coveralls.io/repos/github/Azure/azure-storage-ruby/badge.svg?branch=master)](https://coveralls.io/github/Azure/azure-storage-ruby?branch=master) * Dev: [![Dev Build Status](https://travis-ci.org/Azure/azure-storage-ruby.svg?branch=dev)](https://travis-ci.org/Azure/azure-storage-ruby/branches) [![Coverage Status](https://coveralls.io/repos/github/Azure/azure-storage-ruby/badge.svg?branch=dev)](https://coveralls.io/github/Azure/azure-storage-ruby?branch=dev) -This project provides a Ruby package that makes it easy to access and manage Microsoft Azure Storage Services. - -# Library Features - -* [Blobs](#blobs) -* [Tables](#tables) -* [Queues](#queues) -* [Files](#files) +This project provides a Ruby package that makes it easy to access and manage Microsoft Azure Storage File Services. # Supported Ruby Versions -* Ruby 2.0 -* Ruby 2.1 -* Ruby 2.2 +* Ruby 1.9.3 to 2.5 Note: * x64 Ruby for Windows is known to have some compatibility issues. -* azure-storage depends on gem nokogiri. For Ruby version lower than 2.2, please install the compatible nokogiri before trying to install azure-storage. +* azure-storage-file depends on gem nokogiri. For Ruby version lower than 2.2, please install the compatible nokogiri before trying to install azure-storage-file. # Getting Started ## Install the rubygem package -You can install the azure rubygem package directly. +You can install the azure storage file rubygem package directly. ```bash -gem install azure-storage --pre +gem install azure-storage-file ``` ## Setup Connection -You can use this SDK against the Microsoft Azure Storage Services in the cloud, or against the local Storage Emulator if you are on Windows. +You can use this Client Library against the Microsoft Azure Storage File Services in the cloud. There are two ways you can set up the connections: @@ -49,48 +40,31 @@ There are two ways you can set up the connections: ```ruby - require 'azure/storage' + require 'azure/storage/file' - # Setup a specific instance of an Azure::Storage::Client - client = Azure::Storage::Client.create(:storage_account_name => 'your account name', :storage_access_key => 'your access key') + # Setup a specific instance of an Azure::Storage::File::FileService + file_client = Azure::Storage::File::FileService.create(storage_account_name: 'your account name', storage_access_key: 'your access key') - # Or create a client and store as a singleton - Azure::Storage.setup(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - # Then you can either call Azure::Storage.client.some_method or Azure::Storage.some_method to invoke a method on the Storage Client + # Or create a client and initialize with it. + require 'azure/storage/common' + common_client = Azure::Storage::Common::Client.create(storage_account_name: 'your account name', storage_access_key: 'your access key') + file_client = Azure::Storage::File::FileService.new(client: common_client) # Configure a ca_cert.pem file if you are having issues with ssl peer verification - client.ca_file = './ca_file.pem' - -``` - -* Against local Emulator (Windows Only) - -```ruby - - require 'azure/storage' - client = Azure::Storage::Client.create_development - - # Or create by options and provide your own proxy_uri - client = Azure::Storage::Client.create(:use_development_storage => true, :development_storage_proxy_uri => 'your proxy uri') + file_client.ca_file = './ca_file.pem' ``` ### Via Environment Variables -* Against Microsoft Azure Storage Services in the cloud +* Against Microsoft Azure Storage File Services in the cloud ```bash export AZURE_STORAGE_ACCOUNT = export AZURE_STORAGE_ACCESS_KEY = ``` -* Against local Emulator (Windows Only) - - ```bash - export EMULATED = true - ``` - * [SSL Certificate File](https://gist.github.com/fnichol/867550) if having issues with ssl peer verification ```bash @@ -99,195 +73,46 @@ There are two ways you can set up the connections: # Usage - -## Blobs - -```ruby - -# Require the azure storage rubygem -require 'azure/storage' - -# Setup a specific instance of an Azure::Storage::Client -client = Azure::Storage::Client.create(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Alternatively, create a client that can anonymously access public containers for read operations -client = Azure::Storage::Client.create(storage_blob_host: "https://youraccountname.blob.core.windows.net") - -# Get an azure storage blob service object from a specific instance of an Azure::Storage::Client -blobs = client.blob_client - -# Or setup the client as a singleton -Azure::Storage.setup(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Create an azure storage blob service object after you set up the credentials -blobs = Azure::Storage::Blob::BlobService.new - -# Add retry filter to the service object -blobs.with_filter(Azure::Storage::Core::Filter::ExponentialRetryPolicyFilter.new) - -# Create a container -container = blobs.create_container('test-container') - -# Upload a Blob -content = ::File.open('test.jpg', 'rb') { |file| file.read } -blobs.create_block_blob(container.name, 'image-blob', content) - -# List containers -blobs.list_containers() - -# List Blobs -blobs.list_blobs(container.name) - -# Download a Blob -blob, content = blobs.get_blob(container.name, 'image-blob') -::File.open('download.png', 'wb') {|f| f.write(content)} - -# Delete a Blob -blobs.delete_blob(container.name, 'image-blob') - -``` - -## Tables - -```ruby - -# Require the azure storage rubygem -require 'azure/storage' - -# Setup a specific instance of an Azure::Storage::Client -client = Azure::Storage::Client.create(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Get an azure storage table service object from a specific instance of an Azure::Storage::Client -tables = client.table_client - -# Or setup the client as a singleton -Azure::Storage.setup(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Create an azure storage table service object after you set up the credentials -tables = Azure::Storage::Table::TableService.new - -# Add retry filter to the service object -tables.with_filter(Azure::Storage::Core::Filter::ExponentialRetryPolicyFilter.new) - -# Create a table -tables.create_table('testtable') - -# Insert an entity -entity = { content: 'test entity', PartitionKey: 'test-partition-key', RowKey: '1' } -tables.insert_entity('testtable', entity) - -# Get an entity -result = tables.get_entity('testtable', 'test-partition-key', '1') - -# Update an entity -result.properties['content'] = 'test entity with updated content' -tables.update_entity(result.table, result.properties) - -# Query entities -query = { :filter => "content eq 'test entity'" } -result, token = tables.query_entities('testtable', query) - -# Delete an entity -tables.delete_entity('testtable', 'test-partition-key', '1') - -# delete a table -tables.delete_table('testtable') - -``` - - -## Queues - -```ruby - -# Require the azure storage rubygem -require 'azure/storage' - -# Setup a specific instance of an Azure::Storage::Client -client = Azure::Storage::Client.create(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Get an azure storage queue service object from a specific instance of an Azure::Storage::Client -queues = client.queue_client - -# Or setup the client as a singleton -Azure::Storage.setup(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Create an azure storage queue service object after you set up the credentials -queues = Azure::Storage::Queue::QueueService.new - -# Add retry filter to the service object -queues.with_filter(Azure::Storage::Core::Filter::ExponentialRetryPolicyFilter.new) - -# Create a queue -queues.create_queue('test-queue') - -# Create a message -queues.create_message('test-queue', 'test message') - -# Get one or more messages with setting the visibility timeout -result = queues.list_messages('test-queue', 30, { number_of_messages: 10 }) - -# Get one or more messages without setting the visibility timeout -result = queues.peek_messages('test-queue', { number_of_messages: 10 }) - -# Update a message -message = queues.list_messages('test-queue', 30) -pop_receipt, time_next_visible = queues.update_message('test-queue', message[0].id, message[0].pop_receipt, 'updated test message', 30) - -# Delete a message -message = queues.list_messages('test-queue', 30) -queues.delete_message('test-queue', message[0].id, message[0].pop_receipt) - -# Delete a queue -queues.delete_queue('test-queue') - -``` - ## Files ```ruby -# Require the azure storage rubygem -require 'azure/storage' - -# Setup a specific instance of an Azure::Storage::Client -client = Azure::Storage::Client.create(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Get an azure storage file service object from a specific instance of an Azure::Storage::Client -files = client.file_client +# Require the azure storage file rubygem +require 'azure/storage/file' -# Or setup the client as a singleton -Azure::Storage.setup(:storage_account_name => 'your account name', :storage_access_key => 'your access key') +# Setup a specific instance of an Azure::Storage::File::FileService +client = Azure::Storage::File::FileService.create(:storage_account_name => 'your account name', :storage_access_key => 'your access key') -# Create an azure storage file service object after you set up the credentials -files = Azure::Storage::File::FileService.new +# Alternatively, create a client that can anonymously access public containers for read operations +client = Azure::Storage::File::FileService.create(storage_file_host: "https://youraccountname.file.core.windows.net") # Add retry filter to the service object -files.with_filter(Azure::Storage::Core::Filter::ExponentialRetryPolicyFilter.new) +require "azure/storage/common" +client.with_filter(Azure::Storage::Common::Core::Filter::ExponentialRetryPolicyFilter.new) # Create a share -share = files.create_share('test-share') +share = client.create_share('test-share') # Create a directory -directory = files.create_directory(share.name, 'test-directory') +directory = client.create_directory(share.name, 'test-directory') # Create a file and update the file content content = ::File.open('test.jpg', 'rb') { |file| file.read } -file = files.create_file(share.name, directory.name, 'test-file', content.size) -files.put_file_range(share.name, directory.name, file.name, 0, content.size - 1, content) +file = client.create_file(share.name, directory.name, 'test-file', content.size) +client.put_file_range(share.name, directory.name, file.name, 0, content.size - 1, content) # List shares -files.list_shares() +client.list_shares() -# List directories and files -files.list_directories_and_files(share.name, directory.name) +# List directories and client +client.list_directories_and_files(share.name, directory.name) # Download a File -file, content = files.get_file(share.name, directory.name, file.name) +file, content = client.get_file(share.name, directory.name, file.name) ::File.open('download.png', 'wb') {|f| f.write(content)} # Delete a File -files.delete_file(share.name, directory.name, file.name) +client.delete_file(share.name, directory.name, file.name) ``` @@ -297,32 +122,11 @@ You can customize the user-agent string by setting your user agent prefix when c ```ruby # Require the azure storage rubygem -require "azure/storage" +require "azure/storage/file" # Setup a specific instance of an Azure::Storage::Client with :user_agent_prefix option -client = Azure::Storage::Client.create(:storage_account_name => "your account name", :storage_access_key => "your access key", :user_agent_prefix => "your application name") +client = Azure::Storage::File::FileService.create(:storage_account_name => "your account name", :storage_access_key => "your access key", :user_agent_prefix => "your application name") ``` -# Getting Started for Contributors - -If you would like to become an active contributor to this project please follow the instructions provided in [Azure Projects Contribution Guidelines](http://azure.github.io/guidelines/). -You can find more details for contributing in the [CONTRIBUTING.md](CONTRIBUTING.md). - -# Provide Feedback - -If you encounter any bugs with the library please file an issue in the [Issues](https://github.com/Azure/azure-storage-ruby/issues) section of the project. - -# Azure Storage SDKs and Tooling - -* [Azure Storage Client Library for .Net](http://github.com/azure/azure-storage-net) -* [Azure Storage Client Library for Java](http://github.com/azure/azure-storage-java) -* [Azure Storage Client Library for Node.js](http://github.com/azure/azure-storage-node) -* [Azure Storage Client Library for Python](http://github.com/azure/azure-storage-python) -* [Azure Storage Client Library for Ruby](http://github.com/azure/azure-storage-ruby) -* [Azure Storage Client Library for C++](http://github.com/azure/azure-storage-cpp) -* [Azure Storage Client Library for iOS](http://github.com/azure/azure-storage-ios) -* [Azure Storage Client Library for Android](http://github.com/azure/azure-storage-android) -* [Azure Storage Data Movement Library](https://github.com/Azure/azure-storage-net-data-movement) - # Code of Conduct This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments. diff --git a/file/lib/azure/storage/file/file.rb b/file/lib/azure/storage/file/file.rb index ed37294f..636d2f0d 100644 --- a/file/lib/azure/storage/file/file.rb +++ b/file/lib/azure/storage/file/file.rb @@ -653,7 +653,7 @@ def abort_copy_file(share, directory_path, file, copy_id, options = {}) # See https://docs.microsoft.com/en-us/rest/api/storageservices/fileservices/create-file # # Returns a File - def create_file_with_content(share, directory_path, file, length, content, options = {}) + def create_file_from_content(share, directory_path, file, length, content, options = {}) options[:content_type] = get_or_apply_content_type(content, options[:content_type]) create_file(share, directory_path, file, length, options) diff --git a/file/lib/azure/storage/file/file_service.rb b/file/lib/azure/storage/file/file_service.rb index 47feddcf..66060bd4 100644 --- a/file/lib/azure/storage/file/file_service.rb +++ b/file/lib/azure/storage/file/file_service.rb @@ -138,12 +138,14 @@ def create_from_connection_string(connection_string, &block) # * +:use_path_style_uri+ - String. Whether use path style URI for specified endpoints # * +:ca_file+ - String. File path of the CA file if having issue with SSL # * +:user_agent_prefix+ - String. The user agent prefix that can identify the application calls the library + # * +:client+ - Azure::Storage::Common::Client. The common client used to initalize the service. # # The valid set of options include: # * Storage Emulator: +:use_development_storage+ required, +:development_storage_proxy_uri+ optionally # * Storage account name and key: +:storage_account_name+ and +:storage_access_key+ required, set +:storage_dns_suffix+ necessarily # * Storage account name and SAS token: +:storage_account_name+ and +:storage_sas_token+ required, set +:storage_dns_suffix+ necessarily # * Specified hosts and SAS token: At least one of the service host and SAS token. It's up to user to ensure the SAS token is suitable for the serivce + # * Azure::Storage::Common::Client: The common client used to initalize the service. This client can be initalized and used repeatedly. # * Anonymous File: only +:storage_file_host+, if it is to only access files within a container # # Additional notes: diff --git a/queue/BreakingChanges.md b/queue/BreakingChanges.md index 94b645f9..b0783a5d 100644 --- a/queue/BreakingChanges.md +++ b/queue/BreakingChanges.md @@ -1,32 +1,4 @@ -Tracking Breaking Changes in 0.14.0-preview +Tracking Breaking Changes in 1.0.0 -QUEUE -* The return type of `Azure::Storage::Queue::create_message` is changed from `nil` to an `Azure::Storage::Queue::Message` object. - -Tracking Breaking Changes in 0.13.0-preview - -TABLE -* The return type `Azure::Service::EnumerationResult` of `query_tables` has a changed structure. Now the `'updated'` will not be contained, and is flattened to a structure in the form of `{ {"TableName" => "tableone"}, {"TableName" => "tabletwo"}, {"TableName" => "tablethree"}}`. -* The `Azure::Storage::Table::Entity` does not contain `:table` and `updated` anymore. The updated time can be found in `:properties`. -* The return type of `get_table` is changed to a Hash that contains full metadata returned from the server when query the table. -* The method `Azure::Storage::Table::EdmType::unserialize_query_value` is renamed to `deserialize_value`. - -Tracking Breaking Changes in 0.11.0-preview - -ALL -* `Azure::Storage::Core::Auth::SharedAccessSignature.generate` is renamed to `generate_service_sas_token`. -* `Azure::Storage::Core::Auth::SharedAccessSignature.signed_uri` requires `use_account_sas` as the second parameter. -* Removed the support for the Shared Key Lite. - -Tracking Breaking Changes in 0.10.0-preview - -ALL -* Require "azure-storage" instead of "azure_storage". - -BLOB -* The `create_blob_pages` method is renamed to `put_blob_pages`. -* The `create_blob_block` method is renamed to `put_blob_block`. -* The `acquire_lease` method is renamed to `acquire_blob_lease`. -* The `renew_lease` method is renamed to `renew_blob_lease`. -* The `release_lease` method is renamed to `release_blob_lease`. -* The `break_lease` method is renamed to `break_blob_lease`. +* This module now only consists of functionalities to access Azure Storage Queue Service. +* Creating Queue Client using `Azure::Storage::Client.create` is now deprecated. To create a Queue client, users have to choose from `Azure::Storage::Queue::QueueService::create`, `Azure::Storage::Queue::QueueService::create_development`, ``Azure::Storage::Queue::QueueService::create_from_env`, `Azure::Storage::Queue::QueueService::create_from_connection_string` or `Azure::Storage::Queue::QueueService.new`. The parameters remain unchanged. diff --git a/queue/CONTRIBUTING.md b/queue/CONTRIBUTING.md deleted file mode 100644 index 9db89891..00000000 --- a/queue/CONTRIBUTING.md +++ /dev/null @@ -1,74 +0,0 @@ -If you intend to contribute to the project, please make sure you've followed the instructions provided in the [Azure Projects Contribution Guidelines](http://azure.github.io/guidelines/). -## Project Setup -The Azure Storage development team uses Visual Studio Code so instructions will be tailored to that preference. However, any preferred IDE or other toolset should be usable. - -### Install -* Ruby 1.9.3, 2.0, 2.1 or 2.2. -* [Visual Studio Code](https://code.visualstudio.com/) - -### Development Environment Setup -To get the source code of the SDK via **git** just type: - -```bash -git clone https://github.com/Azure/azure-storage-ruby.git -cd ./azure-storage-ruby -``` - -Then, install Bundler and run bundler to install all the gem dependencies: - -```bash -gem install bundler -bundle install -``` - -## Tests - -### Configuration -If you would like to run the integration test suite, you will need to setup environment variables which will be used -during the integration tests. These tests will use these credentials to run live tests against Azure with the provided -credentials (you will be charged for usage, so verify the clean up scripts did their job at the end of a test run). - -The root of the project contains a [.env_sample](.env_sample) file. This dot file is a sample of the actual environment vars needed to -run the integration tests. - -Unit tests don't require real credentials and don't require to provide the .env file. - -Do the following to prepare your environment for integration tests: - -* Copy [.env_sample](.env_sample) to .env **relative to root of the project dir** -* Update .env with your credentials **.env is in the .gitignore, so should only reside locally** - -### Running -You can use the following commands to run: - -* All the tests: ``rake test``. **This will run integration tests if you have .env file or env vars setup** -* Run storage suite of tests: ``rake test:unit``, ``rake test:integration`` -* One particular test file: ``ruby -I"lib:test" ""`` - -### Testing Features -As you develop a feature, you'll need to write tests to ensure quality. Your changes should be covered by both unit tests and integration tests. You should also run existing tests related to your change to address any unexpected breaks. - -## Pull Requests - -### Guidelines -The following are the minimum requirements for any pull request that must be met before contributions can be accepted. -* Make sure you've signed the CLA before you start working on any change. -* Discuss any proposed contribution with the team via a GitHub issue **before** starting development. -* Code must be professional quality - * No style issues - * You should strive to mimic the style with which we have written the library - * Clean, well-commented, well-designed code - * Try to limit the number of commits for a feature to 1-2. If you end up having too many we may ask you to squash your changes into fewer commits. -* [ChangeLog.md](ChangeLog.md) needs to be updated describing the new change -* Thoroughly test your feature - -### Branching Policy -Changes should be based on the **dev** branch, not master as master is considered publicly released code. Each breaking change should be recorded in [BreakingChanges.md](BreakingChanges.md). - -### Adding Features for All Platforms -We strive to release each new feature for each of our environments at the same time. Therefore, we ask that all contributions be written for Ruby 1.9.3 and later. - -### Review Process -We expect all guidelines to be met before accepting a pull request. As such, we will work with you to address issues we find by leaving comments in your code. Please understand that it may take a few iterations before the code is accepted as we maintain high standards on code quality. Once we feel comfortable with a contribution, we will validate the change and accept the pull request. - -Thank you for any contributions! Please let the team know if you have any questions or concerns about our contribution policy. \ No newline at end of file diff --git a/queue/ChangeLog.md b/queue/ChangeLog.md index 54232455..984cbe10 100644 --- a/queue/ChangeLog.md +++ b/queue/ChangeLog.md @@ -1,207 +1,5 @@ -2017.12 - version 1.0.0 +2018.1 - version 1.0.0 -BLOB -* Added following convenience APIs to support large payload upload from local to append or page blob. - - Azure::Storage::Blob::BlobService::create_page_blob_with_content - - Azure::Storage::Blob::BlobService::create_append_blob_from_content -* Added the support for `Azure::Storage::Blob::BlobService::create_block_blob` to handle large payload that used to require making multiple `Azure::Storage::Blob::BlobService::put_blob_block` calls and calling `Azure::Storage::Blob::BlobService::commit_blob_blocks`. - -File -* Added convenience API `Azure::Storage::File::FileService::create_file_with_content` to support large payload upload from local to file. - -2017.11 - version 0.15.0-preview - -ALL -* Added the support for the location mode in the API options. -* Added the support for retrying according to the location mode. - -BLOB -* Added the support for retrieving statistics related to replication for the Blob service. -* Added the support for anonymous read access of public containers. -* Added full lease ID header support for following methods: - - Azure::Storage::Blob::BlobService::get_container_properties - - Azure::Storage::Blob::BlobService::get_container_metadata - - Azure::Storage::Blob::BlobService::get_container_acl - - Azure::Storage::Blob::BlobService::set_container_metadata - - Azure::Storage::Blob::BlobService::set_container_acl - - Azure::Storage::Blob::BlobService::delete_container - - Azure::Storage::Blob::BlobService::create_block_blob - - Azure::Storage::Blob::BlobService::create_page_blob - - Azure::Storage::Blob::BlobService::create_append_blob - - Azure::Storage::Blob::BlobService::get_blob - - Azure::Storage::Blob::BlobService::get_blob_properties - - Azure::Storage::Blob::BlobService::set_blob_properties - - Azure::Storage::Blob::BlobService::get_blob_metadata - - Azure::Storage::Blob::BlobService::set_blob_metadata - - Azure::Storage::Blob::BlobService::create_blob_snapshot - - Azure::Storage::Blob::BlobService::copy_blob - - Azure::Storage::Blob::BlobService::copy_blob_from_uri - - Azure::Storage::Blob::BlobService::delete_blob - - Azure::Storage::Blob::BlobService::put_blob_block - - Azure::Storage::Blob::BlobService::commit_blob_blocks - - Azure::Storage::Blob::BlobService::list_blob_blocks - - Azure::Storage::Blob::BlobService::put_blob_pages - - Azure::Storage::Blob::BlobService::list_page_blob_ranges - - Azure::Storage::Blob::BlobService::incremental_copy_blob - - Azure::Storage::Blob::BlobService::append_blob_block - -Queue -* Added the support for retrieving statistics related to replication for the Queue service. - -Table -* Added the support for retrieving statistics related to replication for the Table service. - -2017.09 - version 0.14.0-preview - -ALL -* Added configuration file for Rubocop and auto-resolved coding style issue. - -BLOB -* The `Azure::Storage::Blob::list_page_blob_ranges` API now accepts `:previous_snapshot` as an optional parameter, that specifies that the response returns pages that have been updated or cleared since the snapshot specified by `:previous_snapshot` was taken. -* The `Azure::Storage::Blob::Blob` object now has an attribute `:encrypted` showing if the blob or blob related request has been encrypted. -* The `Azure::Storage::Blob::BlobService::list_containers` and `Azure::Storage::Blob::BlobService::get_container_properties` will now also return public access level for each container. -* The stored Content-MD5 property is now returned when requesting a range of a blob. Previously this was only returned for full blob downloads. `Azure::Storage::Blob::Blob.properties[:content_md5]` will always hold the stored Content_MD5 property, and `Azure::Storage::Blob::Blob.properties[:range_md5]` will always represent the MD5 for the content returned from the server. -* Added an API `Azure::Storage::Blob::BlobService::incremental_copy_blob` to support [incremental copy](https://docs.microsoft.com/en-us/rest/api/storageservices/incremental-copy-blob) for page blob snapshots. - -FILE -* The stored Content-MD5 property is now returned when requesting a range of a file. Previously this was only returned for full file downloads. `Azure::Storage::File::File.properties[:content_md5]` will always hold the stored Content_MD5 property, and `Azure::Storage::File::File.properties[:range_md5]` will always represent the MD5 for the content returned from the server. - -QUEUE -* The return type of `Azure::Storage::Queue::create_message` is changed from `nil` to an `Azure::Storage::Queue::Message` object. - -FILE -* The API `Azure::Storage::File::list_directories_and_files` now also accepts `:prefix` as an optional parameter. The return value will be filtered with the specified prefix if set. - -2017.09 - version 0.13.0-preview - -ALL -* Removed Nokogiri from Gemfile because it causes bundler fail to install azure-storage. Added it back to runtime dependency and explicitly require user to install the correct version of Nokogiri in README.md. -* Service version is upgraded to 2016-05-31. - -BLOB -* Block size can now be up to 100MB. - -TABLE -* The return type `Azure::Service::EnumerationResult` of `query_tables` has a changed structure. Now the `'updated'` will not be contained, and is flattened to a structure in the form of `{ {"TableName" => "tableone"}, {"TableName" => "tabletwo"}, {"TableName" => "tablethree"}}`. -* The `Azure::Storage::Table::Entity` does not contain `:table` and `updated` anymore. The updated time can be found in `:properties`. -* The return type of `get_table` is changed to a Hash that contains full metadata returned from the server when query the table. -* The method `Azure::Storage::Table::EdmType::unserialize_query_value` is renamed to `deserialize_value`. - -2017.08 - version 0.12.3-preview - -ALL -* Added Nokogiri as a gem into Gemfile, resolving an issue where bundler failed to recognize that the dependency exists after installation. - -2017.08 - version 0.12.2-preview - -ALL -* Removed Nokogiri as a dependency to resolve conflict version caused by azure-core also depending on Nokogiri. - -2017.04 - version 0.12.1-preview - -ALL -* Relaxed constraint on Nokogiri version dependency to allow Nokogiri 1.7.x for Ruby 2.1 and later, but preserving support for Ruby 1.9.3. (Note that this may require updating to Bundler 1.13 or later if you're using Ruby 1.9 or 2.0.) - -2017.02 - version 0.12.0-preview - -ALL -* Fixed the issue where `should_retry?` in the retry_filter.rb overwrites the result from derived `apply_retry_policy`. [#76](https://github.com/Azure/azure-storage-ruby/issues/76) -* Fixed the issue where `Azure::Storage::Client.create_from_connection_string` throws an exception. [#77](https://github.com/Azure/azure-storage-ruby/issues/77) -* Added the support for setting the "timeout" option in `get_service_properties` and `set_service_properties`. - -BLOB -* Added the metadata to the returning instance when creates a blob. -* Added `transactional_md5` to the options of `put_blob_pages`. - -FILE -* Added File Service support, targeting storage service version 2015-04-05. - -2016.12 - version 0.11.5-preview - -ALL -* Added the support for setting customer user agent. [#71](https://github.com/Azure/azure-storage-ruby/issues/71) -* Added the support for hooking in sending requests. - -2016.11 - version 0.11.4-preview - -ALL -* Removed the unnecessary dependencies. [#55](https://github.com/Azure/azure-storage-ruby/issues/55), [#67](https://github.com/Azure/azure-storage-ruby/issues/67) - -BLOB -* Fixed the issue when checking the content encoding. -* Fixed the wrong "Content-Encoding" value in the test cases. -* Fixed the issue where it cannot use the `create_block_blob` method with an IO/File object. [#61](https://github.com/Azure/azure-storage-ruby/issues/61) - -2016.10 - version 0.11.3-preview - -ALL -* Fixed an issue in retry policies. - -2016.10 - version 0.11.2-preview - -ALL -* Fixed the issue where it retries on HTTP 4xx errors. - -BLOB -* Fixed the issue of wrong "Content-Encoding". [#49](https://github.com/Azure/azure-storage-ruby/issues/49) - -2016.09 - version 0.11.1-preview - -ALL -* Added the support for setting the client request ID via the "request_id" parameter. -* Added the retry for the timeout errors. -* Added the retry for the connection reset error. - -BLOB -* Fixed the issue where "list_blobs" doesn't work when delimiter is specified. [#41](https://github.com/Azure/azure-storage-ruby/issues/41) - -2016.08 - version 0.11.0-preview - -ALL -* Added the support for the account shared access signature. -* Removed the support for the Shared Key Lite. - -BLOB -* Added the support for the "add" and "create" permissions in the blob service shared access signature. - -FILE -* Added the support for the "create" permission in the file service shared access signature. - -2016.06 - version 0.10.2-preview - -ALL -* Fixed the issue that cannot run against storage emulator on Windows. -* Fixed the issue that it doesn't run as a singleton when it calls Azure::Storage.setup. -* Updated to storage service version 2015-04-05. - -2016.05 - version 0.10.1-preview - -ALL -* Replaced the core module by the 'azure-core' gem. -* Stopped maintaining the support for Ruby 1.9. -* Added support for retry filters. - -2016.03 - version 0.10.0-preview - -ALL -* Separated out parts of Azure Storage previously found in the Azure SDK 0.7.0 to establish an independent release cycle. -* Supported Ruby 1.9.3, 2.0, 2.1 and 2.2. -* Updated to storage service version 2015-02-21. -* Fixed issue where previous query's parameters were used, causing authentication to fail (https://github.com/Azure/azure-sdk-for-ruby/issues/276). -* Fixed the issue that the Content-MD5 is overwritten when request body exists, regardless the input value. -* Refined the code for setting service properties to be compatible with the XML schema. - -BLOB -* Added support for container lease operations. -* Added support for changing the ID of an existing lease. -* Added support for copying from source blob URI. -* Added support for aborting copying a blob. -* Added support for creating an append blob. -* Added support for appending a block to an appendblob. -* Added support for the content disposition property of a blob. -* Added support for resizing a page blob. -* Added support for setting a page blob's sequence number. -* Fixed the issue where conditional headers for some APIs could not be set. -* Fixed the issue where the request fails when calling list_page_blob_ranges with start range and end range. -* Renamed create_blob_block to put_blob_block. -* Renamed create_blob_pages to put_blob_pages. +* This module now only consists of functionalities to access Azure Storage Queue Service. +* Creating Queue Client using `Azure::Storage::Client.create` is now deprecated. To create a Queue client, users have to choose from `Azure::Storage::Queue::QueueService::create`, `Azure::Storage::Queue::QueueService::create_development`, ``Azure::Storage::Queue::QueueService::create_from_env`, `Azure::Storage::Queue::QueueService::create_from_connection_string` or `Azure::Storage::Queue::QueueService.new`. The parameters remain unchanged. +* Resolved an issue where decoding messages could throw unexpected exception when message text is empty. diff --git a/queue/README.md b/queue/README.md index 3c553121..8479dd94 100644 --- a/queue/README.md +++ b/queue/README.md @@ -1,42 +1,33 @@ # Microsoft Azure Storage Queue Client Library for Ruby -[![Gem Version](https://badge.fury.io/rb/azure-storage.svg)](https://badge.fury.io/rb/azure-storage) +[![Gem Version](https://badge.fury.io/rb/azure-storage-queue.svg)](https://badge.fury.io/rb/azure-storage-queue) * Master: [![Master Build Status](https://travis-ci.org/Azure/azure-storage-ruby.svg?branch=master)](https://travis-ci.org/Azure/azure-storage-ruby/branches) [![Coverage Status](https://coveralls.io/repos/github/Azure/azure-storage-ruby/badge.svg?branch=master)](https://coveralls.io/github/Azure/azure-storage-ruby?branch=master) * Dev: [![Dev Build Status](https://travis-ci.org/Azure/azure-storage-ruby.svg?branch=dev)](https://travis-ci.org/Azure/azure-storage-ruby/branches) [![Coverage Status](https://coveralls.io/repos/github/Azure/azure-storage-ruby/badge.svg?branch=dev)](https://coveralls.io/github/Azure/azure-storage-ruby?branch=dev) -This project provides a Ruby package that makes it easy to access and manage Microsoft Azure Storage Services. - -# Library Features - -* [Blobs](#blobs) -* [Tables](#tables) -* [Queues](#queues) -* [Files](#files) +This project provides a Ruby package that makes it easy to access and manage Microsoft Azure Storage Queue Services. # Supported Ruby Versions -* Ruby 2.0 -* Ruby 2.1 -* Ruby 2.2 +* Ruby 1.9.3 to 2.5 Note: * x64 Ruby for Windows is known to have some compatibility issues. -* azure-storage depends on gem nokogiri. For Ruby version lower than 2.2, please install the compatible nokogiri before trying to install azure-storage. +* azure-storage-queue depends on gem nokogiri. For Ruby version lower than 2.2, please install the compatible nokogiri before trying to install azure-storage-queue. # Getting Started ## Install the rubygem package -You can install the azure rubygem package directly. +You can install the azure storage queue rubygem package directly. ```bash -gem install azure-storage --pre +gem install azure-storage-queue ``` ## Setup Connection -You can use this SDK against the Microsoft Azure Storage Services in the cloud, or against the local Storage Emulator if you are on Windows. +You can use this Client Library against the Microsoft Azure Queue Storage Services in the cloud, or against the local Storage Emulator if you are on Windows. There are two ways you can set up the connections: @@ -49,17 +40,18 @@ There are two ways you can set up the connections: ```ruby - require 'azure/storage' + require 'azure/storage/queue' - # Setup a specific instance of an Azure::Storage::Client - client = Azure::Storage::Client.create(:storage_account_name => 'your account name', :storage_access_key => 'your access key') + # Setup a specific instance of an Azure::Storage::Queue::QueueService + queue_client = Azure::Storage::Queue::QueueService.create(storage_account_name: 'your account name', storage_access_key: 'your access key') - # Or create a client and store as a singleton - Azure::Storage.setup(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - # Then you can either call Azure::Storage.client.some_method or Azure::Storage.some_method to invoke a method on the Storage Client + # Or create a client and initialize with it. + require 'azure/storage/common' + common_client = Azure::Storage::Common::Client.create(storage_account_name: 'your account name', storage_access_key: 'your access key') + queue_client = Azure::Storage::Queue::QueueService.new(client: common_client) # Configure a ca_cert.pem file if you are having issues with ssl peer verification - client.ca_file = './ca_file.pem' + queue_client.ca_file = './ca_file.pem' ``` @@ -67,18 +59,18 @@ There are two ways you can set up the connections: ```ruby - require 'azure/storage' - client = Azure::Storage::Client.create_development + require 'azure/storage/queue' + client = Azure::Storage::Queue::QueueService.create_development # Or create by options and provide your own proxy_uri - client = Azure::Storage::Client.create(:use_development_storage => true, :development_storage_proxy_uri => 'your proxy uri') + client = Azure::Storage::Queue::QueueService.create(:use_development_storage => true, :development_storage_proxy_uri => 'your proxy uri') ``` ### Via Environment Variables -* Against Microsoft Azure Storage Services in the cloud +* Against Microsoft Azure Storage Queue Services in the cloud ```bash export AZURE_STORAGE_ACCOUNT = @@ -99,230 +91,61 @@ There are two ways you can set up the connections: # Usage - -## Blobs - -```ruby - -# Require the azure storage rubygem -require 'azure/storage' - -# Setup a specific instance of an Azure::Storage::Client -client = Azure::Storage::Client.create(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Alternatively, create a client that can anonymously access public containers for read operations -client = Azure::Storage::Client.create(storage_blob_host: "https://youraccountname.blob.core.windows.net") - -# Get an azure storage blob service object from a specific instance of an Azure::Storage::Client -blobs = client.blob_client - -# Or setup the client as a singleton -Azure::Storage.setup(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Create an azure storage blob service object after you set up the credentials -blobs = Azure::Storage::Blob::BlobService.new - -# Add retry filter to the service object -blobs.with_filter(Azure::Storage::Core::Filter::ExponentialRetryPolicyFilter.new) - -# Create a container -container = blobs.create_container('test-container') - -# Upload a Blob -content = ::File.open('test.jpg', 'rb') { |file| file.read } -blobs.create_block_blob(container.name, 'image-blob', content) - -# List containers -blobs.list_containers() - -# List Blobs -blobs.list_blobs(container.name) - -# Download a Blob -blob, content = blobs.get_blob(container.name, 'image-blob') -::File.open('download.png', 'wb') {|f| f.write(content)} - -# Delete a Blob -blobs.delete_blob(container.name, 'image-blob') - -``` - -## Tables - -```ruby - -# Require the azure storage rubygem -require 'azure/storage' - -# Setup a specific instance of an Azure::Storage::Client -client = Azure::Storage::Client.create(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Get an azure storage table service object from a specific instance of an Azure::Storage::Client -tables = client.table_client - -# Or setup the client as a singleton -Azure::Storage.setup(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Create an azure storage table service object after you set up the credentials -tables = Azure::Storage::Table::TableService.new - -# Add retry filter to the service object -tables.with_filter(Azure::Storage::Core::Filter::ExponentialRetryPolicyFilter.new) - -# Create a table -tables.create_table('testtable') - -# Insert an entity -entity = { content: 'test entity', PartitionKey: 'test-partition-key', RowKey: '1' } -tables.insert_entity('testtable', entity) - -# Get an entity -result = tables.get_entity('testtable', 'test-partition-key', '1') - -# Update an entity -result.properties['content'] = 'test entity with updated content' -tables.update_entity(result.table, result.properties) - -# Query entities -query = { :filter => "content eq 'test entity'" } -result, token = tables.query_entities('testtable', query) - -# Delete an entity -tables.delete_entity('testtable', 'test-partition-key', '1') - -# delete a table -tables.delete_table('testtable') - -``` - ## Queues ```ruby -# Require the azure storage rubygem -require 'azure/storage' - -# Setup a specific instance of an Azure::Storage::Client -client = Azure::Storage::Client.create(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Get an azure storage queue service object from a specific instance of an Azure::Storage::Client -queues = client.queue_client +# Require the azure storage queue rubygem +require 'azure/storage/queue' -# Or setup the client as a singleton -Azure::Storage.setup(:storage_account_name => 'your account name', :storage_access_key => 'your access key') +# Setup a specific instance of an Azure::Storage::Queue::QueueService +client = Azure::Storage::Queue::QueueService.create(:storage_account_name => 'your account name', :storage_access_key => 'your access key') -# Create an azure storage queue service object after you set up the credentials -queues = Azure::Storage::Queue::QueueService.new +# Alternatively, create a client that can anonymously access public containers for read operations +client = Azure::Storage::Queue::QueueService.create(storage_queue_host: "https://youraccountname.queue.core.windows.net") # Add retry filter to the service object -queues.with_filter(Azure::Storage::Core::Filter::ExponentialRetryPolicyFilter.new) +require "azure/storage/common" +client.with_filter(Azure::Storage::Common::Core::Filter::ExponentialRetryPolicyFilter.new) # Create a queue -queues.create_queue('test-queue') +client.create_queue('test-queue') # Create a message -queues.create_message('test-queue', 'test message') +client.create_message('test-queue', 'test message') # Get one or more messages with setting the visibility timeout -result = queues.list_messages('test-queue', 30, { number_of_messages: 10 }) +result = client.list_messages('test-queue', 30, { number_of_messages: 10 }) # Get one or more messages without setting the visibility timeout -result = queues.peek_messages('test-queue', { number_of_messages: 10 }) +result = client.peek_messages('test-queue', { number_of_messages: 10 }) # Update a message -message = queues.list_messages('test-queue', 30) -pop_receipt, time_next_visible = queues.update_message('test-queue', message[0].id, message[0].pop_receipt, 'updated test message', 30) +message = client.list_messages('test-queue', 30) +pop_receipt, time_next_visible = client.update_message('test-queue', message[0].id, message[0].pop_receipt, 'updated test message', 30) # Delete a message -message = queues.list_messages('test-queue', 30) -queues.delete_message('test-queue', message[0].id, message[0].pop_receipt) +message = client.list_messages('test-queue', 30) +client.delete_message('test-queue', message[0].id, message[0].pop_receipt) # Delete a queue -queues.delete_queue('test-queue') +client.delete_queue('test-queue') ``` - -## Files - -```ruby -# Require the azure storage rubygem -require 'azure/storage' - -# Setup a specific instance of an Azure::Storage::Client -client = Azure::Storage::Client.create(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Get an azure storage file service object from a specific instance of an Azure::Storage::Client -files = client.file_client - -# Or setup the client as a singleton -Azure::Storage.setup(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Create an azure storage file service object after you set up the credentials -files = Azure::Storage::File::FileService.new - -# Add retry filter to the service object -files.with_filter(Azure::Storage::Core::Filter::ExponentialRetryPolicyFilter.new) - -# Create a share -share = files.create_share('test-share') - -# Create a directory -directory = files.create_directory(share.name, 'test-directory') - -# Create a file and update the file content -content = ::File.open('test.jpg', 'rb') { |file| file.read } -file = files.create_file(share.name, directory.name, 'test-file', content.size) -files.put_file_range(share.name, directory.name, file.name, 0, content.size - 1, content) - -# List shares -files.list_shares() - -# List directories and files -files.list_directories_and_files(share.name, directory.name) - -# Download a File -file, content = files.get_file(share.name, directory.name, file.name) -::File.open('download.png', 'wb') {|f| f.write(content)} - -# Delete a File -files.delete_file(share.name, directory.name, file.name) -``` - ## Customize the user-agent You can customize the user-agent string by setting your user agent prefix when creating the service client. ```ruby -# Require the azure storage rubygem -require "azure/storage" +# Require the azure storage queue rubygem +require "azure/storage/queue" # Setup a specific instance of an Azure::Storage::Client with :user_agent_prefix option -client = Azure::Storage::Client.create(:storage_account_name => "your account name", :storage_access_key => "your access key", :user_agent_prefix => "your application name") +client = Azure::Storage::Queue::QueueService.create(:storage_account_name => "your account name", :storage_access_key => "your access key", :user_agent_prefix => "your application name") ``` -# Getting Started for Contributors - -If you would like to become an active contributor to this project please follow the instructions provided in [Azure Projects Contribution Guidelines](http://azure.github.io/guidelines/). -You can find more details for contributing in the [CONTRIBUTING.md](CONTRIBUTING.md). - -# Provide Feedback - -If you encounter any bugs with the library please file an issue in the [Issues](https://github.com/Azure/azure-storage-ruby/issues) section of the project. - -# Azure Storage SDKs and Tooling - -* [Azure Storage Client Library for .Net](http://github.com/azure/azure-storage-net) -* [Azure Storage Client Library for Java](http://github.com/azure/azure-storage-java) -* [Azure Storage Client Library for Node.js](http://github.com/azure/azure-storage-node) -* [Azure Storage Client Library for Python](http://github.com/azure/azure-storage-python) -* [Azure Storage Client Library for Ruby](http://github.com/azure/azure-storage-ruby) -* [Azure Storage Client Library for C++](http://github.com/azure/azure-storage-cpp) -* [Azure Storage Client Library for iOS](http://github.com/azure/azure-storage-ios) -* [Azure Storage Client Library for Android](http://github.com/azure/azure-storage-android) -* [Azure Storage Data Movement Library](https://github.com/Azure/azure-storage-net-data-movement) - # Code of Conduct This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments. diff --git a/queue/lib/azure/storage/queue/queue_service.rb b/queue/lib/azure/storage/queue/queue_service.rb index 7c98c493..ff6ed191 100644 --- a/queue/lib/azure/storage/queue/queue_service.rb +++ b/queue/lib/azure/storage/queue/queue_service.rb @@ -129,12 +129,14 @@ def create_from_connection_string(connection_string, &block) # * +:use_path_style_uri+ - String. Whether use path style URI for specified endpoints # * +:ca_file+ - String. File path of the CA file if having issue with SSL # * +:user_agent_prefix+ - String. The user agent prefix that can identify the application calls the library + # * +:client+ - Azure::Storage::Common::Client. The common client used to initalize the service. # # The valid set of options include: # * Storage Emulator: +:use_development_storage+ required, +:development_storage_proxy_uri+ optionally # * Storage account name and key: +:storage_account_name+ and +:storage_access_key+ required, set +:storage_dns_suffix+ necessarily # * Storage account name and SAS token: +:storage_account_name+ and +:storage_sas_token+ required, set +:storage_dns_suffix+ necessarily # * Specified hosts and SAS token: At least one of the service host and SAS token. It's up to user to ensure the SAS token is suitable for the serivce + # * Azure::Storage::Common::Client: The common client used to initalize the service. This client can be initalized and used repeatedly. # * Anonymous Queue: only +:storage_queue_host+, if it is to only access queues within a container # # Additional notes: diff --git a/table/BreakingChanges.md b/table/BreakingChanges.md index 94b645f9..850f41f4 100644 --- a/table/BreakingChanges.md +++ b/table/BreakingChanges.md @@ -1,32 +1,4 @@ -Tracking Breaking Changes in 0.14.0-preview +Tracking Breaking Changes in 1.0.0 -QUEUE -* The return type of `Azure::Storage::Queue::create_message` is changed from `nil` to an `Azure::Storage::Queue::Message` object. - -Tracking Breaking Changes in 0.13.0-preview - -TABLE -* The return type `Azure::Service::EnumerationResult` of `query_tables` has a changed structure. Now the `'updated'` will not be contained, and is flattened to a structure in the form of `{ {"TableName" => "tableone"}, {"TableName" => "tabletwo"}, {"TableName" => "tablethree"}}`. -* The `Azure::Storage::Table::Entity` does not contain `:table` and `updated` anymore. The updated time can be found in `:properties`. -* The return type of `get_table` is changed to a Hash that contains full metadata returned from the server when query the table. -* The method `Azure::Storage::Table::EdmType::unserialize_query_value` is renamed to `deserialize_value`. - -Tracking Breaking Changes in 0.11.0-preview - -ALL -* `Azure::Storage::Core::Auth::SharedAccessSignature.generate` is renamed to `generate_service_sas_token`. -* `Azure::Storage::Core::Auth::SharedAccessSignature.signed_uri` requires `use_account_sas` as the second parameter. -* Removed the support for the Shared Key Lite. - -Tracking Breaking Changes in 0.10.0-preview - -ALL -* Require "azure-storage" instead of "azure_storage". - -BLOB -* The `create_blob_pages` method is renamed to `put_blob_pages`. -* The `create_blob_block` method is renamed to `put_blob_block`. -* The `acquire_lease` method is renamed to `acquire_blob_lease`. -* The `renew_lease` method is renamed to `renew_blob_lease`. -* The `release_lease` method is renamed to `release_blob_lease`. -* The `break_lease` method is renamed to `break_blob_lease`. +* This module now only consists of functionalities to access Azure Storage Table Service. +* Creating Table Client using `Azure::Storage::Client.create` is now deprecated. To create a Table client, users have to choose from `Azure::Storage::Table::TableService::create`, `Azure::Storage::Table::TableService::create_development`, ``Azure::Storage::Table::TableService::create_from_env`, `Azure::Storage::Table::TableService::create_from_connection_string` or `Azure::Storage::Table::TableService.new`. The parameters remain unchanged. diff --git a/table/CONTRIBUTING.md b/table/CONTRIBUTING.md deleted file mode 100644 index 9db89891..00000000 --- a/table/CONTRIBUTING.md +++ /dev/null @@ -1,74 +0,0 @@ -If you intend to contribute to the project, please make sure you've followed the instructions provided in the [Azure Projects Contribution Guidelines](http://azure.github.io/guidelines/). -## Project Setup -The Azure Storage development team uses Visual Studio Code so instructions will be tailored to that preference. However, any preferred IDE or other toolset should be usable. - -### Install -* Ruby 1.9.3, 2.0, 2.1 or 2.2. -* [Visual Studio Code](https://code.visualstudio.com/) - -### Development Environment Setup -To get the source code of the SDK via **git** just type: - -```bash -git clone https://github.com/Azure/azure-storage-ruby.git -cd ./azure-storage-ruby -``` - -Then, install Bundler and run bundler to install all the gem dependencies: - -```bash -gem install bundler -bundle install -``` - -## Tests - -### Configuration -If you would like to run the integration test suite, you will need to setup environment variables which will be used -during the integration tests. These tests will use these credentials to run live tests against Azure with the provided -credentials (you will be charged for usage, so verify the clean up scripts did their job at the end of a test run). - -The root of the project contains a [.env_sample](.env_sample) file. This dot file is a sample of the actual environment vars needed to -run the integration tests. - -Unit tests don't require real credentials and don't require to provide the .env file. - -Do the following to prepare your environment for integration tests: - -* Copy [.env_sample](.env_sample) to .env **relative to root of the project dir** -* Update .env with your credentials **.env is in the .gitignore, so should only reside locally** - -### Running -You can use the following commands to run: - -* All the tests: ``rake test``. **This will run integration tests if you have .env file or env vars setup** -* Run storage suite of tests: ``rake test:unit``, ``rake test:integration`` -* One particular test file: ``ruby -I"lib:test" ""`` - -### Testing Features -As you develop a feature, you'll need to write tests to ensure quality. Your changes should be covered by both unit tests and integration tests. You should also run existing tests related to your change to address any unexpected breaks. - -## Pull Requests - -### Guidelines -The following are the minimum requirements for any pull request that must be met before contributions can be accepted. -* Make sure you've signed the CLA before you start working on any change. -* Discuss any proposed contribution with the team via a GitHub issue **before** starting development. -* Code must be professional quality - * No style issues - * You should strive to mimic the style with which we have written the library - * Clean, well-commented, well-designed code - * Try to limit the number of commits for a feature to 1-2. If you end up having too many we may ask you to squash your changes into fewer commits. -* [ChangeLog.md](ChangeLog.md) needs to be updated describing the new change -* Thoroughly test your feature - -### Branching Policy -Changes should be based on the **dev** branch, not master as master is considered publicly released code. Each breaking change should be recorded in [BreakingChanges.md](BreakingChanges.md). - -### Adding Features for All Platforms -We strive to release each new feature for each of our environments at the same time. Therefore, we ask that all contributions be written for Ruby 1.9.3 and later. - -### Review Process -We expect all guidelines to be met before accepting a pull request. As such, we will work with you to address issues we find by leaving comments in your code. Please understand that it may take a few iterations before the code is accepted as we maintain high standards on code quality. Once we feel comfortable with a contribution, we will validate the change and accept the pull request. - -Thank you for any contributions! Please let the team know if you have any questions or concerns about our contribution policy. \ No newline at end of file diff --git a/table/ChangeLog.md b/table/ChangeLog.md index 54232455..f87a6a48 100644 --- a/table/ChangeLog.md +++ b/table/ChangeLog.md @@ -1,207 +1,6 @@ -2017.12 - version 1.0.0 +2018.1 - version 1.0.0 -BLOB -* Added following convenience APIs to support large payload upload from local to append or page blob. - - Azure::Storage::Blob::BlobService::create_page_blob_with_content - - Azure::Storage::Blob::BlobService::create_append_blob_from_content -* Added the support for `Azure::Storage::Blob::BlobService::create_block_blob` to handle large payload that used to require making multiple `Azure::Storage::Blob::BlobService::put_blob_block` calls and calling `Azure::Storage::Blob::BlobService::commit_blob_blocks`. - -File -* Added convenience API `Azure::Storage::File::FileService::create_file_with_content` to support large payload upload from local to file. - -2017.11 - version 0.15.0-preview - -ALL -* Added the support for the location mode in the API options. -* Added the support for retrying according to the location mode. - -BLOB -* Added the support for retrieving statistics related to replication for the Blob service. -* Added the support for anonymous read access of public containers. -* Added full lease ID header support for following methods: - - Azure::Storage::Blob::BlobService::get_container_properties - - Azure::Storage::Blob::BlobService::get_container_metadata - - Azure::Storage::Blob::BlobService::get_container_acl - - Azure::Storage::Blob::BlobService::set_container_metadata - - Azure::Storage::Blob::BlobService::set_container_acl - - Azure::Storage::Blob::BlobService::delete_container - - Azure::Storage::Blob::BlobService::create_block_blob - - Azure::Storage::Blob::BlobService::create_page_blob - - Azure::Storage::Blob::BlobService::create_append_blob - - Azure::Storage::Blob::BlobService::get_blob - - Azure::Storage::Blob::BlobService::get_blob_properties - - Azure::Storage::Blob::BlobService::set_blob_properties - - Azure::Storage::Blob::BlobService::get_blob_metadata - - Azure::Storage::Blob::BlobService::set_blob_metadata - - Azure::Storage::Blob::BlobService::create_blob_snapshot - - Azure::Storage::Blob::BlobService::copy_blob - - Azure::Storage::Blob::BlobService::copy_blob_from_uri - - Azure::Storage::Blob::BlobService::delete_blob - - Azure::Storage::Blob::BlobService::put_blob_block - - Azure::Storage::Blob::BlobService::commit_blob_blocks - - Azure::Storage::Blob::BlobService::list_blob_blocks - - Azure::Storage::Blob::BlobService::put_blob_pages - - Azure::Storage::Blob::BlobService::list_page_blob_ranges - - Azure::Storage::Blob::BlobService::incremental_copy_blob - - Azure::Storage::Blob::BlobService::append_blob_block - -Queue -* Added the support for retrieving statistics related to replication for the Queue service. - -Table -* Added the support for retrieving statistics related to replication for the Table service. - -2017.09 - version 0.14.0-preview - -ALL -* Added configuration file for Rubocop and auto-resolved coding style issue. - -BLOB -* The `Azure::Storage::Blob::list_page_blob_ranges` API now accepts `:previous_snapshot` as an optional parameter, that specifies that the response returns pages that have been updated or cleared since the snapshot specified by `:previous_snapshot` was taken. -* The `Azure::Storage::Blob::Blob` object now has an attribute `:encrypted` showing if the blob or blob related request has been encrypted. -* The `Azure::Storage::Blob::BlobService::list_containers` and `Azure::Storage::Blob::BlobService::get_container_properties` will now also return public access level for each container. -* The stored Content-MD5 property is now returned when requesting a range of a blob. Previously this was only returned for full blob downloads. `Azure::Storage::Blob::Blob.properties[:content_md5]` will always hold the stored Content_MD5 property, and `Azure::Storage::Blob::Blob.properties[:range_md5]` will always represent the MD5 for the content returned from the server. -* Added an API `Azure::Storage::Blob::BlobService::incremental_copy_blob` to support [incremental copy](https://docs.microsoft.com/en-us/rest/api/storageservices/incremental-copy-blob) for page blob snapshots. - -FILE -* The stored Content-MD5 property is now returned when requesting a range of a file. Previously this was only returned for full file downloads. `Azure::Storage::File::File.properties[:content_md5]` will always hold the stored Content_MD5 property, and `Azure::Storage::File::File.properties[:range_md5]` will always represent the MD5 for the content returned from the server. - -QUEUE -* The return type of `Azure::Storage::Queue::create_message` is changed from `nil` to an `Azure::Storage::Queue::Message` object. - -FILE -* The API `Azure::Storage::File::list_directories_and_files` now also accepts `:prefix` as an optional parameter. The return value will be filtered with the specified prefix if set. - -2017.09 - version 0.13.0-preview - -ALL -* Removed Nokogiri from Gemfile because it causes bundler fail to install azure-storage. Added it back to runtime dependency and explicitly require user to install the correct version of Nokogiri in README.md. -* Service version is upgraded to 2016-05-31. - -BLOB -* Block size can now be up to 100MB. - -TABLE -* The return type `Azure::Service::EnumerationResult` of `query_tables` has a changed structure. Now the `'updated'` will not be contained, and is flattened to a structure in the form of `{ {"TableName" => "tableone"}, {"TableName" => "tabletwo"}, {"TableName" => "tablethree"}}`. -* The `Azure::Storage::Table::Entity` does not contain `:table` and `updated` anymore. The updated time can be found in `:properties`. -* The return type of `get_table` is changed to a Hash that contains full metadata returned from the server when query the table. -* The method `Azure::Storage::Table::EdmType::unserialize_query_value` is renamed to `deserialize_value`. - -2017.08 - version 0.12.3-preview - -ALL -* Added Nokogiri as a gem into Gemfile, resolving an issue where bundler failed to recognize that the dependency exists after installation. - -2017.08 - version 0.12.2-preview - -ALL -* Removed Nokogiri as a dependency to resolve conflict version caused by azure-core also depending on Nokogiri. - -2017.04 - version 0.12.1-preview - -ALL -* Relaxed constraint on Nokogiri version dependency to allow Nokogiri 1.7.x for Ruby 2.1 and later, but preserving support for Ruby 1.9.3. (Note that this may require updating to Bundler 1.13 or later if you're using Ruby 1.9 or 2.0.) - -2017.02 - version 0.12.0-preview - -ALL -* Fixed the issue where `should_retry?` in the retry_filter.rb overwrites the result from derived `apply_retry_policy`. [#76](https://github.com/Azure/azure-storage-ruby/issues/76) -* Fixed the issue where `Azure::Storage::Client.create_from_connection_string` throws an exception. [#77](https://github.com/Azure/azure-storage-ruby/issues/77) -* Added the support for setting the "timeout" option in `get_service_properties` and `set_service_properties`. - -BLOB -* Added the metadata to the returning instance when creates a blob. -* Added `transactional_md5` to the options of `put_blob_pages`. - -FILE -* Added File Service support, targeting storage service version 2015-04-05. - -2016.12 - version 0.11.5-preview - -ALL -* Added the support for setting customer user agent. [#71](https://github.com/Azure/azure-storage-ruby/issues/71) -* Added the support for hooking in sending requests. - -2016.11 - version 0.11.4-preview - -ALL -* Removed the unnecessary dependencies. [#55](https://github.com/Azure/azure-storage-ruby/issues/55), [#67](https://github.com/Azure/azure-storage-ruby/issues/67) - -BLOB -* Fixed the issue when checking the content encoding. -* Fixed the wrong "Content-Encoding" value in the test cases. -* Fixed the issue where it cannot use the `create_block_blob` method with an IO/File object. [#61](https://github.com/Azure/azure-storage-ruby/issues/61) - -2016.10 - version 0.11.3-preview - -ALL -* Fixed an issue in retry policies. - -2016.10 - version 0.11.2-preview - -ALL -* Fixed the issue where it retries on HTTP 4xx errors. - -BLOB -* Fixed the issue of wrong "Content-Encoding". [#49](https://github.com/Azure/azure-storage-ruby/issues/49) - -2016.09 - version 0.11.1-preview - -ALL -* Added the support for setting the client request ID via the "request_id" parameter. -* Added the retry for the timeout errors. -* Added the retry for the connection reset error. - -BLOB -* Fixed the issue where "list_blobs" doesn't work when delimiter is specified. [#41](https://github.com/Azure/azure-storage-ruby/issues/41) - -2016.08 - version 0.11.0-preview - -ALL -* Added the support for the account shared access signature. -* Removed the support for the Shared Key Lite. - -BLOB -* Added the support for the "add" and "create" permissions in the blob service shared access signature. - -FILE -* Added the support for the "create" permission in the file service shared access signature. - -2016.06 - version 0.10.2-preview - -ALL -* Fixed the issue that cannot run against storage emulator on Windows. -* Fixed the issue that it doesn't run as a singleton when it calls Azure::Storage.setup. -* Updated to storage service version 2015-04-05. - -2016.05 - version 0.10.1-preview - -ALL -* Replaced the core module by the 'azure-core' gem. -* Stopped maintaining the support for Ruby 1.9. -* Added support for retry filters. - -2016.03 - version 0.10.0-preview - -ALL -* Separated out parts of Azure Storage previously found in the Azure SDK 0.7.0 to establish an independent release cycle. -* Supported Ruby 1.9.3, 2.0, 2.1 and 2.2. -* Updated to storage service version 2015-02-21. -* Fixed issue where previous query's parameters were used, causing authentication to fail (https://github.com/Azure/azure-sdk-for-ruby/issues/276). -* Fixed the issue that the Content-MD5 is overwritten when request body exists, regardless the input value. -* Refined the code for setting service properties to be compatible with the XML schema. - -BLOB -* Added support for container lease operations. -* Added support for changing the ID of an existing lease. -* Added support for copying from source blob URI. -* Added support for aborting copying a blob. -* Added support for creating an append blob. -* Added support for appending a block to an appendblob. -* Added support for the content disposition property of a blob. -* Added support for resizing a page blob. -* Added support for setting a page blob's sequence number. -* Fixed the issue where conditional headers for some APIs could not be set. -* Fixed the issue where the request fails when calling list_page_blob_ranges with start range and end range. -* Renamed create_blob_block to put_blob_block. -* Renamed create_blob_pages to put_blob_pages. +* This module now only consists of functionalities to access Azure Storage Table Service. +* Creating Table Client using `Azure::Storage::Client.create` is now deprecated. To create a Table client, users have to choose from `Azure::Storage::Table::TableService::create`, `Azure::Storage::Table::TableService::create_development`, ``Azure::Storage::Table::TableService::create_from_env`, `Azure::Storage::Table::TableService::create_from_connection_string` or `Azure::Storage::Table::TableService.new`. The parameters remain unchanged. +* Added support for Batch operation to perform a single `QueryEntity` operation. +* Resolved an issue where users used connection string to initialize Table Service and use batch operation would fail. diff --git a/table/README.md b/table/README.md index 1d9fd5e6..dc91a69b 100644 --- a/table/README.md +++ b/table/README.md @@ -1,42 +1,33 @@ # Microsoft Azure Storage Table Client Library for Ruby -[![Gem Version](https://badge.fury.io/rb/azure-storage.svg)](https://badge.fury.io/rb/azure-storage) +[![Gem Version](https://badge.fury.io/rb/azure-storage-table.svg)](https://badge.fury.io/rb/azure-storage-table) * Master: [![Master Build Status](https://travis-ci.org/Azure/azure-storage-ruby.svg?branch=master)](https://travis-ci.org/Azure/azure-storage-ruby/branches) [![Coverage Status](https://coveralls.io/repos/github/Azure/azure-storage-ruby/badge.svg?branch=master)](https://coveralls.io/github/Azure/azure-storage-ruby?branch=master) * Dev: [![Dev Build Status](https://travis-ci.org/Azure/azure-storage-ruby.svg?branch=dev)](https://travis-ci.org/Azure/azure-storage-ruby/branches) [![Coverage Status](https://coveralls.io/repos/github/Azure/azure-storage-ruby/badge.svg?branch=dev)](https://coveralls.io/github/Azure/azure-storage-ruby?branch=dev) -This project provides a Ruby package that makes it easy to access and manage Microsoft Azure Storage Services. - -# Library Features - -* [Blobs](#blobs) -* [Tables](#tables) -* [Queues](#queues) -* [Files](#files) +This project provides a Ruby package that makes it easy to access and manage Microsoft Azure Storage Table Services. # Supported Ruby Versions -* Ruby 2.0 -* Ruby 2.1 -* Ruby 2.2 +* Ruby 1.9.3 to 2.5 Note: * x64 Ruby for Windows is known to have some compatibility issues. -* azure-storage depends on gem nokogiri. For Ruby version lower than 2.2, please install the compatible nokogiri before trying to install azure-storage. +* azure-storage-table depends on gem nokogiri. For Ruby version lower than 2.2, please install the compatible nokogiri before trying to install azure-storage-table. # Getting Started ## Install the rubygem package -You can install the azure rubygem package directly. +You can install the azure storage table rubygem package directly. ```bash -gem install azure-storage --pre +gem install azure-storage-table ``` ## Setup Connection -You can use this SDK against the Microsoft Azure Storage Services in the cloud, or against the local Storage Emulator if you are on Windows. +You can use this Client Library against the Microsoft Azure Storage Table Services in the cloud, or against the local Storage Emulator if you are on Windows. There are two ways you can set up the connections: @@ -49,17 +40,18 @@ There are two ways you can set up the connections: ```ruby - require 'azure/storage' + require 'azure/storage/table' - # Setup a specific instance of an Azure::Storage::Client - client = Azure::Storage::Client.create(:storage_account_name => 'your account name', :storage_access_key => 'your access key') + # Setup a specific instance of an Azure::Storage::Table::TableService + table_client = Azure::Storage::Table::TableService.create(storage_account_name: 'your account name', storage_access_key: 'your access key') - # Or create a client and store as a singleton - Azure::Storage.setup(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - # Then you can either call Azure::Storage.client.some_method or Azure::Storage.some_method to invoke a method on the Storage Client + # Or create a client and initialize with it. + require 'azure/storage/common' + common_client = Azure::Storage::Common::Client.create(storage_account_name: 'your account name', storage_access_key: 'your access key') + table_client = Azure::Storage::Table::TableService.new(client: common_client) # Configure a ca_cert.pem file if you are having issues with ssl peer verification - client.ca_file = './ca_file.pem' + table_client.ca_file = './ca_file.pem' ``` @@ -67,18 +59,18 @@ There are two ways you can set up the connections: ```ruby - require 'azure/storage' - client = Azure::Storage::Client.create_development + require 'azure/storage/table' + client = Azure::Storage::Table::TableService.create_development # Or create by options and provide your own proxy_uri - client = Azure::Storage::Client.create(:use_development_storage => true, :development_storage_proxy_uri => 'your proxy uri') + client = Azure::Storage::Table::TableService.create(:use_development_storage => true, :development_storage_proxy_uri => 'your proxy uri') ``` ### Via Environment Variables -* Against Microsoft Azure Storage Services in the cloud +* Against Microsoft Azure Storage Table Services in the cloud ```bash export AZURE_STORAGE_ACCOUNT = @@ -99,53 +91,6 @@ There are two ways you can set up the connections: # Usage - -## Blobs - -```ruby - -# Require the azure storage rubygem -require 'azure/storage' - -# Setup a specific instance of an Azure::Storage::Client -client = Azure::Storage::Client.create(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Alternatively, create a client that can anonymously access public containers for read operations -client = Azure::Storage::Client.create(storage_blob_host: "https://youraccountname.blob.core.windows.net") - -# Get an azure storage blob service object from a specific instance of an Azure::Storage::Client -blobs = client.blob_client - -# Or setup the client as a singleton -Azure::Storage.setup(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Create an azure storage blob service object after you set up the credentials -blobs = Azure::Storage::Blob::BlobService.new - -# Add retry filter to the service object -blobs.with_filter(Azure::Storage::Core::Filter::ExponentialRetryPolicyFilter.new) - -# Create a container -container = blobs.create_container('test-container') - -# Upload a Blob -content = ::File.open('test.jpg', 'rb') { |file| file.read } -blobs.create_block_blob(container.name, 'image-blob', content) - -# List containers -blobs.list_containers() - -# List Blobs -blobs.list_blobs(container.name) - -# Download a Blob -blob, content = blobs.get_blob(container.name, 'image-blob') -::File.open('download.png', 'wb') {|f| f.write(content)} - -# Delete a Blob -blobs.delete_blob(container.name, 'image-blob') - -``` ## Tables @@ -195,134 +140,18 @@ tables.delete_table('testtable') ``` - -## Queues - -```ruby - -# Require the azure storage rubygem -require 'azure/storage' - -# Setup a specific instance of an Azure::Storage::Client -client = Azure::Storage::Client.create(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Get an azure storage queue service object from a specific instance of an Azure::Storage::Client -queues = client.queue_client - -# Or setup the client as a singleton -Azure::Storage.setup(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Create an azure storage queue service object after you set up the credentials -queues = Azure::Storage::Queue::QueueService.new - -# Add retry filter to the service object -queues.with_filter(Azure::Storage::Core::Filter::ExponentialRetryPolicyFilter.new) - -# Create a queue -queues.create_queue('test-queue') - -# Create a message -queues.create_message('test-queue', 'test message') - -# Get one or more messages with setting the visibility timeout -result = queues.list_messages('test-queue', 30, { number_of_messages: 10 }) - -# Get one or more messages without setting the visibility timeout -result = queues.peek_messages('test-queue', { number_of_messages: 10 }) - -# Update a message -message = queues.list_messages('test-queue', 30) -pop_receipt, time_next_visible = queues.update_message('test-queue', message[0].id, message[0].pop_receipt, 'updated test message', 30) - -# Delete a message -message = queues.list_messages('test-queue', 30) -queues.delete_message('test-queue', message[0].id, message[0].pop_receipt) - -# Delete a queue -queues.delete_queue('test-queue') - -``` - - -## Files - -```ruby -# Require the azure storage rubygem -require 'azure/storage' - -# Setup a specific instance of an Azure::Storage::Client -client = Azure::Storage::Client.create(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Get an azure storage file service object from a specific instance of an Azure::Storage::Client -files = client.file_client - -# Or setup the client as a singleton -Azure::Storage.setup(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Create an azure storage file service object after you set up the credentials -files = Azure::Storage::File::FileService.new - -# Add retry filter to the service object -files.with_filter(Azure::Storage::Core::Filter::ExponentialRetryPolicyFilter.new) - -# Create a share -share = files.create_share('test-share') - -# Create a directory -directory = files.create_directory(share.name, 'test-directory') - -# Create a file and update the file content -content = ::File.open('test.jpg', 'rb') { |file| file.read } -file = files.create_file(share.name, directory.name, 'test-file', content.size) -files.put_file_range(share.name, directory.name, file.name, 0, content.size - 1, content) - -# List shares -files.list_shares() - -# List directories and files -files.list_directories_and_files(share.name, directory.name) - -# Download a File -file, content = files.get_file(share.name, directory.name, file.name) -::File.open('download.png', 'wb') {|f| f.write(content)} - -# Delete a File -files.delete_file(share.name, directory.name, file.name) -``` - ## Customize the user-agent You can customize the user-agent string by setting your user agent prefix when creating the service client. ```ruby -# Require the azure storage rubygem -require "azure/storage" +# Require the azure storage table rubygem +require "azure/storage/table" # Setup a specific instance of an Azure::Storage::Client with :user_agent_prefix option -client = Azure::Storage::Client.create(:storage_account_name => "your account name", :storage_access_key => "your access key", :user_agent_prefix => "your application name") +client = Azure::Storage::Table::TableService.create(:storage_account_name => "your account name", :storage_access_key => "your access key", :user_agent_prefix => "your application name") ``` -# Getting Started for Contributors - -If you would like to become an active contributor to this project please follow the instructions provided in [Azure Projects Contribution Guidelines](http://azure.github.io/guidelines/). -You can find more details for contributing in the [CONTRIBUTING.md](CONTRIBUTING.md). - -# Provide Feedback - -If you encounter any bugs with the library please file an issue in the [Issues](https://github.com/Azure/azure-storage-ruby/issues) section of the project. - -# Azure Storage SDKs and Tooling - -* [Azure Storage Client Library for .Net](http://github.com/azure/azure-storage-net) -* [Azure Storage Client Library for Java](http://github.com/azure/azure-storage-java) -* [Azure Storage Client Library for Node.js](http://github.com/azure/azure-storage-node) -* [Azure Storage Client Library for Python](http://github.com/azure/azure-storage-python) -* [Azure Storage Client Library for Ruby](http://github.com/azure/azure-storage-ruby) -* [Azure Storage Client Library for C++](http://github.com/azure/azure-storage-cpp) -* [Azure Storage Client Library for iOS](http://github.com/azure/azure-storage-ios) -* [Azure Storage Client Library for Android](http://github.com/azure/azure-storage-android) -* [Azure Storage Data Movement Library](https://github.com/Azure/azure-storage-net-data-movement) - # Code of Conduct This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments. diff --git a/table/lib/azure/storage/table/table_service.rb b/table/lib/azure/storage/table/table_service.rb index fc09a22b..655eb316 100644 --- a/table/lib/azure/storage/table/table_service.rb +++ b/table/lib/azure/storage/table/table_service.rb @@ -131,12 +131,14 @@ def create_from_connection_string(connection_string, &block) # * +:use_path_style_uri+ - String. Whether use path style URI for specified endpoints # * +:ca_file+ - String. File path of the CA file if having issue with SSL # * +:user_agent_prefix+ - String. The user agent prefix that can identify the application calls the library + # * +:client+ - Azure::Storage::Common::Client. The common client used to initalize the service. # # The valid set of options include: # * Storage Emulator: +:use_development_storage+ required, +:development_storage_proxy_uri+ optionally # * Storage account name and key: +:storage_account_name+ and +:storage_access_key+ required, set +:storage_dns_suffix+ necessarily # * Storage account name and SAS token: +:storage_account_name+ and +:storage_sas_token+ required, set +:storage_dns_suffix+ necessarily # * Specified hosts and SAS token: At least one of the service host and SAS token. It's up to user to ensure the SAS token is suitable for the serivce + # * Azure::Storage::Common::Client: The common client used to initalize the service. This client can be initalized and used repeatedly. # * Anonymous Table: only +:storage_table_host+, if it is to only access tables within a container # # Additional notes: diff --git a/test/integration/blob/append_blob_test.rb b/test/integration/blob/append_blob_test.rb index 3ddfed0d..2899bc1e 100644 --- a/test/integration/blob/append_blob_test.rb +++ b/test/integration/blob/append_blob_test.rb @@ -100,7 +100,7 @@ def eof? length = 4 * 1024 * 1024 content = SecureRandom.random_bytes(length) blob_name = BlobNameHelper.name - subject.create_page_blob_with_content container_name, blob_name, length, content + subject.create_page_blob_from_content container_name, blob_name, length, content blob, body = subject.get_blob(container_name, blob_name) blob.name.must_equal blob_name blob.properties[:content_length].must_equal length diff --git a/test/integration/blob/blob_pages_test.rb b/test/integration/blob/blob_pages_test.rb index 4d0f29c2..0aea63e5 100644 --- a/test/integration/blob/blob_pages_test.rb +++ b/test/integration/blob/blob_pages_test.rb @@ -42,13 +42,13 @@ subject.create_page_blob container_name, blob_name3, length } - describe "#create_page_blob_with_content" do + describe "#create_page_blob_from_content" do it "1MB string payload works" do length = 1 * 1024 * 1024 content = SecureRandom.random_bytes(length) content.force_encoding "utf-8" blob_name = BlobNameHelper.name - subject.create_page_blob_with_content container_name, blob_name, length, content + subject.create_page_blob_from_content container_name, blob_name, length, content blob, body = subject.get_blob(container_name, blob_name) blob.name.must_equal blob_name blob.properties[:content_length].must_equal length @@ -60,7 +60,7 @@ length = 4 * 1024 * 1024 content = SecureRandom.random_bytes(length) blob_name = BlobNameHelper.name - subject.create_page_blob_with_content container_name, blob_name, length, content + subject.create_page_blob_from_content container_name, blob_name, length, content blob, body = subject.get_blob(container_name, blob_name) blob.name.must_equal blob_name blob.properties[:content_length].must_equal length @@ -72,7 +72,7 @@ length = 5 * 1024 * 1024 content = SecureRandom.random_bytes(length) blob_name = BlobNameHelper.name - subject.create_page_blob_with_content container_name, blob_name, length, content + subject.create_page_blob_from_content container_name, blob_name, length, content blob, body = subject.get_blob(container_name, blob_name) blob.name.must_equal blob_name blob.properties[:content_length].must_equal length @@ -87,7 +87,7 @@ file = File.open blob_name, "w+" file.write content file.seek 0 - subject.create_page_blob_with_content container_name, blob_name, length, file + subject.create_page_blob_from_content container_name, blob_name, length, file blob, body = subject.get_blob(container_name, blob_name) blob.name.must_equal blob_name blob.properties[:content_length].must_equal length diff --git a/test/integration/file/create_file_test.rb b/test/integration/file/create_file_test.rb index b8a3e81e..8e89361f 100644 --- a/test/integration/file/create_file_test.rb +++ b/test/integration/file/create_file_test.rb @@ -28,7 +28,7 @@ subject { Azure::Storage::File::FileService.create(SERVICE_CREATE_OPTIONS()) } after { ShareNameHelper.clean } - describe "#create_file_with_content" do + describe "#create_file_from_content" do let(:share_name) { ShareNameHelper.name } let(:directory_name) { FileNameHelper.name } let(:file_name) { FileNameHelper.name } @@ -42,7 +42,7 @@ content = SecureRandom.random_bytes(length) content.force_encoding "utf-8" file_name = FileNameHelper.name - subject.create_file_with_content share_name, directory_name, file_name, length, content + subject.create_file_from_content share_name, directory_name, file_name, length, content file, body = subject.get_file(share_name, directory_name, file_name) file.name.must_equal file_name file.properties[:content_length].must_equal length @@ -54,7 +54,7 @@ length = 4 * 1024 * 1024 content = SecureRandom.random_bytes(length) file_name = FileNameHelper.name - subject.create_file_with_content share_name, directory_name, file_name, length, content + subject.create_file_from_content share_name, directory_name, file_name, length, content file, body = subject.get_file(share_name, directory_name, file_name) file.name.must_equal file_name file.properties[:content_length].must_equal length @@ -66,7 +66,7 @@ length = 5 * 1024 * 1024 content = SecureRandom.random_bytes(length) file_name = FileNameHelper.name - subject.create_file_with_content share_name, directory_name, file_name, length, content + subject.create_file_from_content share_name, directory_name, file_name, length, content file, body = subject.get_file(share_name, directory_name, file_name) file.name.must_equal file_name file.properties[:content_length].must_equal length @@ -81,7 +81,7 @@ local_file = File.open file_name, "w+" local_file.write content local_file.seek 0 - subject.create_file_with_content share_name, directory_name, file_name, length, local_file + subject.create_file_from_content share_name, directory_name, file_name, length, local_file file, body = subject.get_file(share_name, directory_name, file_name) file.name.must_equal file_name file.properties[:content_length].must_equal length