Android (Boox): Autocorrect appends instead of replacing

[eleanorkonik]

When using Codemirror 6 on a Boox, the other apps I'm using (pdf annotators, kindle, etc)... when you hand write something into the little OCR box it has built in, it maintains the spaces, which isn't always the intended rendering of the text (since handwriting necessitates linebreaks and it can't immediately tell whether they're intentional). So the Boox has an autocorrect feature built in where you can tell it "no, what I really wanted was for this to all be part of the same paragraph" and it will normally overwrite the erroneous rendering (with all the extra line breaks) with the preferred version.

But Codemirror 6 seems to be appending instead of replacing when it's doing its autocorrect.

I tried to create a video demonstrating what I'm talking about but it's sort of awkward doing it one-handed, and the xodo app I'm using on the left to demonstrate is being a little weird, I'm assuming because of the split screen, sorry about that. But I promise that the behavior I'm seeing from codemirror doesn't occur in any of the other apps that I or anyone else I asked use. It's unlisted on youtube because I couldn't figure out how to compress it enough for github to tolerate, sorry.

Here's a picture of the duplication happening in the obsidian.md program. The video demonstrates on the codemirror.net/6 website, though.

[marijnh]

This rather sounds very similar to other IME issues we've had where the DOM changes occur in multiple steps (first insert some content, then remove the replaced content), but the IME system gets confused when CodeMirror responds to the first change (possibly by normalizing the cursor).

I don't have a device to test this on, and obscure input methods like this don't get high-priority support, so I can't promise if or when I'll look into this.

(If someone else wants to: the first question would be what the DOM changes made by the IME actually look like, by testing in a plain contentEditable element, and then to figure out what CodeMirror is doing in response to the first change, to disrupt this.)

[walter-cavinaw]

The way the OCR text entry works is that you first write a bunch of text and the boox keyboard waits until you are finished writing. Then it converts it to characters and puts it into the editor. After that it creates suggestions and most apps will replace the first text with the selected suggestion.

For codemirror, when you write into the OCR box without splitting it into multiple lines everything works as expected: After the first text is put into the editor it is underlined/selected, and choosing a suggestion will replace the initial text. That is the expected functionality, so no problems there.

The problem with codemirror comes when you split your written text across multiple lines in the OCR box, as demonstrated in the video. The initial text is put into the editor but is not underlined/selected. The cursor just goes to the end, and when you choose a suggestion/correction from the boox keyboard it gets appended, because the text you entered is no longer underlined/selected.

I think codemirror is not handling the 'new line' character the same way as other apps, such as the kindle app (as shown in the video. ) I think codemirror cancels the selection and puts the cursor to the end of the text whenever it receives text with a 'new line' character.

Does this explanation make sense? It is not related to the OCR box on the device, rather the existence of 'new line' characters in the selected input, which breaks the selection and normalizes the cursor.

[marijnh]

Oh, multi-line composition is not something I've ever seen before—and indeed not something the editor is prepared for. Unfortunately browsers don't really provide a way to determine the extent of the composition, and in this case the heuristic of using the text node around the cursor obviously breaks down.

I have to warn you that I'm not seeing this as a high priority issue right now, since it is triggered by very specific and rare behavior, and I don't even have a setup in which I can test it.