Text encode decode

[hhugo]

No description provided.

[hhugo]

I'd like to benchmark this change

[vouillon]

One should create the encoder and the decoder only once and reuse it, and use a fixed buffer for caml_jsstring_of_string.

[hhugo]

One should create the encoder and the decoder only once and reuse it, and use a fixed buffer for caml_jsstring_of_string.

Done

[hhugo]

@vouillon, any idea on how to benchmark this other that micro benchmarks ?

[hhugo]

A quick micro benchmark show a significant slowdown.
master

n: 3, 0.167000
n: 7, 0.268000
n: 10, 0.371000
n: 20, 0.811000
n: 50, 1.989000
n: 100, 3.618000

this PR

n: 3, 1.209000
n: 7, 1.269000
n: 10, 1.354000
n: 20, 1.660000
n: 50, 2.632000
n: 100, 4.578000

[Copilot]

Copilot reviewed 1 out of 1 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (2)

runtime/js/mlBytes.js:642

• The function caml_jsstring_of_string should handle non-ASCII strings correctly. The previous implementation used caml_utf16_of_utf8, which may handle edge cases differently than TextDecoder.

if (jsoo_is_ascii(s)) return s;

runtime/js/mlBytes.js:659

• The function caml_string_of_jsstring should handle non-ASCII strings correctly. The previous implementation used caml_utf8_of_utf16, which may handle edge cases differently than TextEncoder.

if (jsoo_is_ascii(s)) return caml_string_of_jsbytes(s);

[Copilot]

The method toUtf16 should handle ASCII and non-ASCII cases consistently. The previous implementation used caml_utf16_of_utf8 for non-ASCII strings, which may handle surrogate pairs and invalid sequences differently than TextDecoder.

Suggested change

	if (this.t === 9) return this.c;
	return caml_utf16_of_utf8(r);