Add Encoding note to README and CHANGELOG

janlelis · Dec 26, 2024 · 620454c · 620454c
1 parent a23a070
commit 620454c
Show file tree

Hide file tree

Showing 2 changed files with 12 additions and 0 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,5 +1,12 @@
 # CHANGELOG
 
+## 3.1.3 (unreleased)
+
+Better handling of non-UTF-8 strings:
+
+- Data with *BINARY* encoding is interpreted as UTF-8, if possible
+- Use `invalid: :replace` and `undef: :replace` options when converting to UTF-8
+
 ## 3.1.2
 
 - Performance improvements

diff --git a/README.md b/README.md
@@ -71,6 +71,11 @@ Unicode::DisplayWidth.of("·", 1) # => 1
 Unicode::DisplayWidth.of("·", 2) # => 2
 ```
 
+### Encoding Notes
+
+- Data with *BINARY* encoding is interpreted as UTF-8, if possible
+- Non-UTF-8 strings are converted to UTF-8 before measuring, using the [`{invalid: :replace, undef: :replace}`) options](https://ruby-doc.org/3.3.5/encodings_rdoc.html#label-Encoding+Options)
+
 ### Custom Overwrites
 
 You can overwrite how to handle specific code points by passing a hash (or even a proc) as `overwrite:` parameter: