Bug: fix Unicode when saving

[Mori-Yang]

In this issue, use eslint . --fixcan run successfully, but using the eslint extension to fix it when saving will cause incorrect Unicode encoding.

I think this is a problem caused by the extension and hope to be answered and helped.

[dbaeumer]

@Mori-Yang Can you please provide me with a GitHub repository I can clone with a minimal setup that demos what you are seeing.

[Mori-Yang]

Hi @dbaeumer , I created a repository:

https://github.com/Mori-Yang/eslint-extension-test

you can try saving in index.js, the autofix will cause wrong Unicode. Then try using eslint . --fix, it works fine.

[Mori-Yang]

Hi @dbaeumer .
Sorry to bother you, any progress here? I want to know if this problem is caused by the extension.

[dbaeumer]

Thanks for the test case. I can reproduce this and I think it is a bug in the extension. Very likely a problem with the diffing algorithm.

[Mori-Yang]

@dbaeumer If you can tell me some modifications and code locations that maybe I can try, or if you think you can easily fix it, all the better!

[dbaeumer]

The relevant code is here:

vscode-eslint/server/src/eslintServer.ts

Line 748 in c0e7537

const uri = identifier.uri;

I guess it has something to do with diffing here:

vscode-eslint/server/src/eslintServer.ts

Line 790 in c0e7537

const diffs = stringDiff(originalContent, fixedContent, false);

which doesn't handle the unicode characters correctly when computing the edits.

[dbaeumer]

Best you can do is to beug it and see where exactly the problem happens. Ping me if you have more questions.

[Mori-Yang]

Thank you, I'll try.

[Mori-Yang]

Best you can do is to beug it and see where exactly the problem happens. Ping me if you have more questions.

Hi, @dbaeumer . I debugged the code you mentioned and found that there is no problem with the stringDiff function. Now I understand that the cause of the problem may be:
The two unicode characters in the example (🍊:\ud83c\udf4a 🍌:\ud83c\udf4c) have the same leading byte, so the stringDiff function determines that the following bytes need to be replaced, which I think is the right approach.

However, the result is not right. I guess it is because there is a problem when writing, but I don’t know the corresponding code location. I hope you can tell me. If I have any mistakes, please tell me your thoughts.

[dbaeumer]

Have you checked that the right edits are generate here:

vscode-eslint/server/src/eslintServer.ts

Line 793 in c0e7537

result.push({

[dbaeumer]

And thanks a lot for looking into this. Highly appreciated.