Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Utilities to convert to/from utf-8, utf-16. #146

Merged
merged 2 commits into from
Sep 19, 2017

Conversation

fabioz
Copy link
Contributor

@fabioz fabioz commented Sep 16, 2017

First step to solve #134

@fabioz
Copy link
Contributor Author

fabioz commented Sep 16, 2017

As a note, with these differences, I've done a change to just convert indexes back to utf-16 and things almost seem to work.

You can see my changes at:

fabioz/LiClipseText@1d8f88c

So, mostly I just convert the result back to utf-16 (as everything should be dealing with utf-8 internally), now, I have a strange difference in that it appears that the grammar found an intermediary match that it didn't find first, so, I'm not submitting a pull request because of that...

i.e.:
I expected my test to have the same results with chars with and without unicode chars, yet, as you can see in:
fabioz/LiClipseText@1d8f88c#diff-9b217bdbf9f5aee0083ab4831116a634R62

it's not really exactly the same (the assertEquals above is the one without unicode and the below is with unicode -- the result should be the same).

@angelozerr
Copy link
Contributor

Thanks a lot @fabioz for your PR. Let me find time to study it and compare with VSCode.

@angelozerr angelozerr merged commit e0a751a into eclipse-tm4e:master Sep 19, 2017
@angelozerr
Copy link
Contributor

@fabioz I hope I have fixed problem with UTF-16 (see #134)

I have tested #134 and #136 and it seems it works well in Windows context. Please give me feedback.

I'm using your convert methods which works well, Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants