-
Notifications
You must be signed in to change notification settings - Fork 61
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Convert ByteString from/to Int8Array #386
Draft
fzhinkin
wants to merge
1
commit into
develop
Choose a base branch
from
bytestring-js-extensions
base: develop
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -82,3 +82,15 @@ final fun (kotlinx.io.bytestring/ByteStringBuilder).kotlinx.io.bytestring/append | |
| final fun <#A: kotlin.text/Appendable> (kotlin.io.encoding/Base64).kotlinx.io.bytestring/encodeToAppendable(kotlinx.io.bytestring/ByteString, #A, kotlin/Int = ..., kotlin/Int = ...): #A // kotlinx.io.bytestring/encodeToAppendable|[email protected](kotlinx.io.bytestring.ByteString;0:0;kotlin.Int;kotlin.Int){0§<kotlin.text.Appendable>}[0] | ||
| final fun kotlinx.io.bytestring/ByteString(kotlin/ByteArray...): kotlinx.io.bytestring/ByteString // kotlinx.io.bytestring/ByteString|ByteString(kotlin.ByteArray...){}[0] | ||
| final inline fun kotlinx.io.bytestring/buildByteString(kotlin/Int = ..., kotlin/Function1<kotlinx.io.bytestring/ByteStringBuilder, kotlin/Unit>): kotlinx.io.bytestring/ByteString // kotlinx.io.bytestring/buildByteString|buildByteString(kotlin.Int;kotlin.Function1<kotlinx.io.bytestring.ByteStringBuilder,kotlin.Unit>){}[0] | ||
|
|
||
| // Targets: [js, wasmJs] | ||
| final fun (kotlinx.io.bytestring/ByteString).kotlinx.io.bytestring/toArrayBuffer(): org.khronos.webgl/ArrayBuffer // kotlinx.io.bytestring/toArrayBuffer|[email protected](){}[0] | ||
|
|
||
| // Targets: [js, wasmJs] | ||
| final fun (kotlinx.io.bytestring/ByteString).kotlinx.io.bytestring/toInt8Array(): org.khronos.webgl/Int8Array // kotlinx.io.bytestring/toInt8Array|[email protected](){}[0] | ||
|
|
||
| // Targets: [js, wasmJs] | ||
| final fun (org.khronos.webgl/ArrayBuffer).kotlinx.io.bytestring/toByteString(): kotlinx.io.bytestring/ByteString // kotlinx.io.bytestring/toByteString|[email protected](){}[0] | ||
|
|
||
| // Targets: [js, wasmJs] | ||
| final fun (org.khronos.webgl/Int8Array).kotlinx.io.bytestring/toByteString(): kotlinx.io.bytestring/ByteString // kotlinx.io.bytestring/toByteString|[email protected](){}[0] | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| /* | ||
| * Copyright 2017-2024 JetBrains s.r.o. and respective authors and developers. | ||
| * Use of this source code is governed by the Apache 2.0 license that can be found in the LICENCE file. | ||
| */ | ||
|
|
||
| package kotlinx.io.bytestring | ||
|
|
||
| import org.khronos.webgl.ArrayBuffer | ||
| import org.khronos.webgl.Int8Array | ||
|
|
||
| /** | ||
| * Returns a new [Int8Array] instance initialized with bytes copied from [this] ByteString. | ||
| */ | ||
| public fun ByteString.toInt8Array(): Int8Array = toByteArray().unsafeCast<Int8Array>() | ||
|
|
||
| /** | ||
| * Returns a new [ArrayBuffer] instance initialized with bytes copied from [this] ByteString. | ||
| */ | ||
| public fun ByteString.toArrayBuffer(): ArrayBuffer = toInt8Array().buffer | ||
|
|
||
| /** | ||
| * Return a new [ByteString] holding data copied from [this] Int8Array. | ||
| */ | ||
| public fun Int8Array.toByteString(): ByteString = ByteString(*unsafeCast<ByteArray>()) | ||
|
|
||
| /** | ||
| * Returns a new [ByteString] holding data copied from [this] ArrayBuffer | ||
| */ | ||
| public fun ArrayBuffer.toByteString(): ByteString = Int8Array(this).toByteString() | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,70 @@ | ||
| /* | ||
| * Copyright 2017-2024 JetBrains s.r.o. and respective authors and developers. | ||
| * Use of this source code is governed by the Apache 2.0 license that can be found in the LICENCE file. | ||
| */ | ||
|
|
||
| package kotlinx.io.bytestring | ||
|
|
||
| import kotlinx.io.bytestring.unsafe.UnsafeByteStringApi | ||
| import kotlinx.io.bytestring.unsafe.UnsafeByteStringOperations | ||
| import org.khronos.webgl.ArrayBuffer | ||
| import org.khronos.webgl.Int8Array | ||
| import org.khronos.webgl.get | ||
| import org.khronos.webgl.set | ||
| import kotlin.test.Test | ||
| import kotlin.test.assertContentEquals | ||
| import kotlin.test.assertEquals | ||
| import kotlin.test.assertTrue | ||
|
|
||
| class ByteStringConversionsTest { | ||
| @Test | ||
| fun int8ArrayToByteString() { | ||
| assertTrue(Int8Array(0).toByteString().isEmpty()) | ||
|
|
||
| val str = Int8Array(byteArrayOf(1, 2, 3, 4).toTypedArray()).toByteString() | ||
| assertContentEquals(byteArrayOf(1, 2, 3, 4), str.toByteArray()) | ||
| } | ||
|
|
||
| @Test | ||
| fun arrayBufferToByteString() { | ||
| assertTrue(ArrayBuffer(0).toByteString().isEmpty()) | ||
|
|
||
| val str = Int8Array(byteArrayOf(1, 2, 3, 4).toTypedArray()).buffer.toByteString() | ||
| assertContentEquals(byteArrayOf(1, 2, 3, 4), str.toByteArray()) | ||
| } | ||
|
|
||
| @Test | ||
| fun byteStringToInt8Array() { | ||
| assertEquals(0, ByteString().toInt8Array().length) | ||
|
|
||
| val array = ByteString(1, 2, 3, 4).toInt8Array() | ||
| for (idx in 0..<3) { | ||
| assertEquals((idx + 1).toByte(), array[idx], "idx = $idx") | ||
| } | ||
| } | ||
|
|
||
| @Test | ||
| fun byteStringToArrayBuffer() { | ||
| assertEquals(0, ByteString().toArrayBuffer().byteLength) | ||
|
|
||
| val buffer = ByteString(1, 2, 3, 4).toArrayBuffer() | ||
| val array = Int8Array(buffer) | ||
| for (idx in 0..<3) { | ||
| assertEquals((idx + 1).toByte(), array[idx], "idx = $idx") | ||
| } | ||
| } | ||
|
|
||
| @OptIn(UnsafeByteStringApi::class) | ||
| @Test | ||
| fun integrityCheck() { | ||
| val array = Int8Array(byteArrayOf(1, 2, 3, 4).toTypedArray()) | ||
| val str = array.toByteString() | ||
|
|
||
| array[0] = 42 | ||
| assertContentEquals(byteArrayOf(1, 2, 3, 4), str.toByteArray()) | ||
|
|
||
| val array2 = str.toInt8Array() | ||
| UnsafeByteStringOperations.withByteArrayUnsafe(str) { it[1] = 42 } | ||
| assertEquals(2, array2[1]) | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,46 @@ | ||
| /* | ||
| * Copyright 2017-2024 JetBrains s.r.o. and respective authors and developers. | ||
| * Use of this source code is governed by the Apache 2.0 license that can be found in the LICENCE file. | ||
| */ | ||
|
|
||
| package kotlinx.io.bytestring | ||
|
|
||
| import org.khronos.webgl.ArrayBuffer | ||
| import org.khronos.webgl.Int8Array | ||
| import org.khronos.webgl.get | ||
| import org.khronos.webgl.set | ||
|
|
||
| /** | ||
| * Returns a new [Int8Array] instance initialized with bytes copied from [this] ByteString. | ||
| */ | ||
| public fun ByteString.toInt8Array(): Int8Array { | ||
| val res = Int8Array(size) | ||
| for (idx in this.indices) { | ||
| res[idx] = this[idx] | ||
| } | ||
| return res | ||
| } | ||
|
|
||
| /** | ||
| * Returns a new [ArrayBuffer] instance initialized with bytes copied from [this] ByteString. | ||
| */ | ||
| public fun ByteString.toArrayBuffer(): ArrayBuffer = toInt8Array().buffer | ||
|
|
||
| /** | ||
| * Return a new [ByteString] holding data copied from [this] Int8Array. | ||
| */ | ||
| public fun Int8Array.toByteString(): ByteString { | ||
| if (length == 0) { | ||
| return ByteString.EMPTY | ||
| } | ||
| val builder = ByteStringBuilder(length) | ||
| for (idx in 0..<length) { | ||
| builder.append(this[idx]) | ||
| } | ||
| return builder.toByteString() | ||
| } | ||
|
|
||
| /** | ||
| * Returns a new [ByteString] holding data copied from [this] ArrayBuffer | ||
| */ | ||
| public fun ArrayBuffer.toByteString(): ByteString = Int8Array(this).toByteString() |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,78 @@ | ||
| /* | ||
| * Copyright 2017-2024 JetBrains s.r.o. and respective authors and developers. | ||
| * Use of this source code is governed by the Apache 2.0 license that can be found in the LICENCE file. | ||
| */ | ||
|
|
||
| package kotlinx.io.bytestring | ||
|
|
||
| import kotlinx.io.bytestring.unsafe.UnsafeByteStringApi | ||
| import kotlinx.io.bytestring.unsafe.UnsafeByteStringOperations | ||
| import org.khronos.webgl.ArrayBuffer | ||
| import org.khronos.webgl.Int8Array | ||
| import org.khronos.webgl.get | ||
| import org.khronos.webgl.set | ||
| import kotlin.test.Test | ||
| import kotlin.test.assertContentEquals | ||
| import kotlin.test.assertEquals | ||
| import kotlin.test.assertTrue | ||
|
|
||
| private fun int8ArrayOf(vararg bytes: Byte): Int8Array { | ||
| val array = Int8Array(bytes.size) | ||
| for (i in bytes.indices) { | ||
| array[i] = bytes[i] | ||
| } | ||
| return array | ||
| } | ||
|
|
||
| class ByteStringConversionsTest { | ||
| @Test | ||
| fun int8ArrayToByteString() { | ||
| assertTrue(Int8Array(0).toByteString().isEmpty()) | ||
|
|
||
| val str = int8ArrayOf(1, 2, 3, 4).toByteString() | ||
| assertContentEquals(byteArrayOf(1, 2, 3, 4), str.toByteArray()) | ||
| } | ||
|
|
||
| @Test | ||
| fun arrayBufferToByteString() { | ||
| assertTrue(ArrayBuffer(0).toByteString().isEmpty()) | ||
|
|
||
| val str = int8ArrayOf(1, 2, 3, 4).buffer.toByteString() | ||
| assertContentEquals(byteArrayOf(1, 2, 3, 4), str.toByteArray()) | ||
| } | ||
|
|
||
| @Test | ||
| fun byteStringToInt8Array() { | ||
| assertEquals(0, ByteString().toInt8Array().length) | ||
|
|
||
| val array = ByteString(1, 2, 3, 4).toInt8Array() | ||
| for (idx in 0..<3) { | ||
| assertEquals((idx + 1).toByte(), array[idx], "idx = $idx") | ||
| } | ||
| } | ||
|
|
||
| @Test | ||
| fun byteStringToArrayBuffer() { | ||
| assertEquals(0, ByteString().toArrayBuffer().byteLength) | ||
|
|
||
| val buffer = ByteString(1, 2, 3, 4).toArrayBuffer() | ||
| val array = Int8Array(buffer) | ||
| for (idx in 0..<3) { | ||
| assertEquals((idx + 1).toByte(), array[idx], "idx = $idx") | ||
| } | ||
| } | ||
|
|
||
| @OptIn(UnsafeByteStringApi::class) | ||
| @Test | ||
| fun integrityCheck() { | ||
| val array = int8ArrayOf(1, 2, 3, 4) | ||
| val str = array.toByteString() | ||
|
|
||
| array[0] = 42 | ||
| assertContentEquals(byteArrayOf(1, 2, 3, 4), str.toByteArray()) | ||
|
|
||
| val array2 = str.toInt8Array() | ||
| UnsafeByteStringOperations.withByteArrayUnsafe(str) { it[1] = 42 } | ||
| assertEquals(2, array2[1]) | ||
| } | ||
| } |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is something that should probably be addressed for all kotlinx libraries: should everyone use these built-in types, which are pretty weird considering we generally don't want anything to do with WebGL, or should they come from kotlin-wrappers, which tend to be the most used?
The question is important because the only "interop" between same types coming from different libs is unsafe-casting.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If an
ArrayBufferoriginating fromkotlin-wrappersneeds to be converted to/fromByteString, then it should be done either inkotlin-wrappers, or in a separate library.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's reasonable. However I'm thinking that since most people on K/JS are using kotlin-wrappers (and most commonly its
kotlin-jsmodule), it would be a surprise to them that the code is red in the IDE when callingInt8Array.toByteString(), for example.It's a bit of an unfortunate situation because it's all fragmented into multiple places.
The
org.khronos.webglpackage is explicitly documented as a wrapper for WebGL, so the question "why am I using WebGL types?" would come up fairly naturally.The
org.khronos.webglpackage is a remnant of the extraction (KT-35973) of DOM APIs from the stdlib, and there is also an open issue about moving away from it (KT-57303). So maybe using kotlin-wrapper types ensures better stability on the long run.I guess the right call would be to have the Kotlin team officially say "we will all use these types for JS", and I guess being kotlin-wrappers is the most complete and up to date collection, that could be the direction. If the stdlib needs a small interop layer, its
externals could be madeinternal.Anyway, from my perspective, it's all good anyway, I'd just
unsafeCast<T>, but it's something I wanted to mention for more clarity.