Support for IDN (punycode) handles with standard sanitization

[drash-course]

This is a follow up PR to #7043.

Currently, all @handle.tld are displayed in their ASCII form. This is fine for most domains, but some TLDs accept IDN domains and it'd be nice to support them fully. I found at least one other issue #6354 discussing this.

Of course, this opens the can of worms that are IDN homograph attacks.

This PR implements the mechanisms described in Unicode TR39, specifically the Restriction Level 2 "Single Script" based on the data from Unicode 16. If the IDN handle passes the checks, it's displayed in full unicode. If it fails the checks, it's displayed in punycode form like @xn--6frz82g.tld. This is what Firefox does as well.

To do so, I wrote a TS script that reads some reference files published by Unicode and generates a "unicode map" of codepoints that the Bluesky app then uses. When a punycode handle is encountered, we decode it and check if the codepoints are not banned and if they only belong to scripts (e.g. Latin, Cyrillic, Hiragana, etc) that can be mixed according to TR39.

The required files are :

• The allow-list of codepoints that can be used for identifiers, such as domain names (https://www.unicode.org/Public/security/16.0.0/IdentifierStatus.txt)
• The Unicode Character Database (https://www.unicode.org/Public/16.0.0/ucdxml/ucd.all.flat.zip)

When a new Unicode version is published, you can put the updated files in scripts/unicode/Public and rerun compileUnicodeMaps.ts to get the new mappings. Something to check on every few months, though there is little risk if it's done later or not at all. You can also choose which codepoints should be banned on top of what is specified in TR39. I banned the checkmark emojis for example, to be aligned with the display name rules.

The resulting "unicode map" is kinda large (about 1600 ranges of codepoints) so I made sure the checking code is speed optimized. This way the app does not need to e.g. memoize the sanitized handles or other fancy tricks to feel smooth. I wrote in timing_test.txt the timing I got with performance.now() while typing xn--a xn--b xn--c in the search bar, if you want to see exact numbers.

Note that searching for IDN handles with the U-label does not work, but typing the xn-- code works. Something to add in a future PR.

Finally, I also added some positive and negative tests for the things I could think of. I've looked for the tests that Firefox may have for IDNs but couldn't find much. Let me know if you feel this needs more tests.

Feel free to give me pointers if you think something is missing.

[drash-course]

Any news from a maintainer on this? Is there an issue or concern that prevents this PR from moving forward?