Rightmost column of text is invisible on moar

[lapo-luchini]

Windows Terminal version

1.21.2911.0

Windows build number

10.0.26120.0

Other Software

moar 1.27.2 installed via go install github.com/walles/moar@latest (issue 250 there)

Steps to reproduce

This happens both on WSL:

go install github.com/walles/moar@latest
seq -s '-' 10 99 | moar -wrap

and also on cmd:

go install github.com/walles/moar@latest
echo 10-11-12-13-14-15-16-17-18-19-20-21-22-23-24-25-26-27-28-29-30-31-32-33-34-35-36-37-38-39-40-41-42-43-44-45-46-47-48-49-50-51-52-53-54-55-56-57-58-59-60-61-62-63-64-65-66-67-68-69-70-71-72-73-74-75-76-77-78-79-80-81-82-83-84-85-86-87-88-89-90-91-92-93-94-95-96-97-98-99 | moar -wrap

Expected Behavior

Line breaking at the end of the terminal width, continuing on next line:

┌────────────────────────┐ screen width
10-11-12-13-14-15-16-17-18
-19-20-21-22-23-24-25-26-2
7-28-29-30…

Actual Behavior

Last column is blank and is not shown:
e.g.

┌────────────────────────┐ screen width
10-11-12-13-14-15-16-17-1
-19-20-21-22-23-24-25-26-
7-28-29-30…

[walles]

FWIW, moar shows the last column of text fine on all tested non-ConPTY terminals.

Test results for moar showing the last column

• ❌ Windows Terminal (on Windows)
• ❌ VSCode terminal on Windows. On macOS, the VSCode terminal works fine.
• ✅ VSCode terminal on macOS. On Windows, the VSCode terminal does not show the last column.
• ✅ iTerm2 on macOS
• ✅ macOS Terminal
• ✅ Hyper on macOS
• ✅ KiTTY on macOS
• ✅ WezTerm on macOS
• ✅ MobaXTerm on Windows: Wrapping long words eats up one character walles/moar#250 (comment)

[DHowett]

This will be very helpful for a differential diagnosis:

Does moar work properly in WSL under Windows Terminal and VSCode?

If so: there's a chance that it has--or is using a library which has--Windows platform-specific code.

If it does, I suspect that it is--or that library is--using GetConsoleScreenBufferInfo(Ex) and treating srWindow as exclusive when it should be inclusive.

[zadjii-msft]

Also,

• Does it work in conhost.exe (not the Terminal)
• Does it work on Terminal Preview (1.22+, where there was a bit of a rewite of ConPTY)

[walles]

If so: there's a chance that it has--or is using a library which has--Windows platform-specific code.

moar is using Go's term.GetSize() function, ending up in here: https://cs.opensource.google/go/x/term/+/refs/tags/v0.25.0:term_windows.go;l=48-54

func getSize(fd int) (width, height int, err error) {
	var info windows.ConsoleScreenBufferInfo
	if err := windows.GetConsoleScreenBufferInfo(windows.Handle(fd), &info); err != nil {
		return 0, 0, err
	}
	return int(info.Window.Right - info.Window.Left + 1), int(info.Window.Bottom - info.Window.Top + 1), nil
}

If it does, I suspect that it is--or that library is--using GetConsoleScreenBufferInfo(Ex) and treating srWindow as exclusive when it should be inclusive.

Let's say the window is 100 chars wide.

If moar thought the windows was 99 chars wide, moar would wrap too early.

But what's happening is that the last char on the line goes missing.

So I believe moar is getting the right information, it just fails to write chars to the last column (with ConPTY).

@DHowett ^

[DHowett]

Ugh, I missed your comment about WSL earlier. I'm sorry.

[DHowett]

@zadjii-msft It reproduces in preview and canary, as well as conhost.

In both cases I used a block selection to highlight the rightmost edge.

[DHowett]

Got it!

This is what moar is printing to the console:

␛[1;1H␛[m␛[38;2;104;104;104m  1 ␛[38;2;208;208;208m10-11-12-13-14-15-16-17-18-19-20-21-22-23-24-25-26-27-28-29-30-31-32-33-34-35-36-37-38-39-40-41-42-43-44-45-46-47-48␛[m␛[K  ␛[m    ␛[38;2;208;208;208m-49-50-51-52-53-54-55-56-57-58-59-60-61-62-63-64-65-66-67-68-69-70-71-72-73-74-75-76-77-78-79-80-81-82-83-84-85-86-8␛[m␛[K  ␛[m    ␛[38;2;208;208;208m7-88-89-90-91-92-93-94-95-96-97-98-99␛[m␛[K  ␛[m␛[7m---␛[m␛[K  ␛[m␛[K  ␛[m␛[K  ␛[m␛[K  ␛[m␛[K  ␛[m␛[K  ␛[m␛[K  ␛[m␛[K  ␛[m␛[K  ␛[m␛[K  ␛[m␛[K  ␛[m␛[K  ␛[m␛[K  ␛[m␛[K  ␛[m␛[K  ␛[m␛[K  ␛[m␛[K  ␛[m␛[K  ␛[m␛[K  ␛[m␛[K  ␛[m␛[K  ␛[m␛[K  ␛[m␛[K  ␛[m␛[K  ␛[m␛[K  ␛[m␛[K  ␛[m␛[7m1 line  100%  Press 'ESC' / 'q' to exit, '/' to search, '?' for help                                                    ␛[m␛[K

Zooming in on the end of the first line (which is 46-47-48 in my case), I see this:

46-47-48␛[m␛[K  ␛[m
       |   |   \- space
       |   \- clear to end of line
       \- fill the rightmost column

By design, emitting a \e[K (EL 0 Erase to Right) when the cursor is in the deferred last column position clears the last column. 🙂

Now to figure out why moar is emitting \e[K in the deferred wrap position!

EDIT to add NOTE: this does not require ConPTY, and does not appear to be a ConPTY issue. This appears to be an application issue (at the moment.)

[walles]

By design, emitting a \e[K (EL 0 Erase to Right) when the cursor is in the deferred last column position clears the last column. 🙂

I will change this behavior in moar.

Out of curiosity, why this design? When all other terminals seem to handle this differently?

[walles]

Now to figure out why moar is emitting \e[K in the deferred wrap position!

Instead of clearing the screen at the start of a redraw, moar clears to EOL after it's done with rendering all chars on that line.

The hope (not measured) is that this might avoid some flickering during redraws.

[DHowett]

Out of curiosity, why this design?

That's a good question, but I should note that xterm does the same thing:

The former libvte (gnome-terminal et al) maintainer filed a bug on us requesting that we deviate from STD 070, and James--who I look to as the paragon of all terminal emulation knowledge--had this to say: #3177 (comment)

He followed up with #14936, which pertains only to the delayed wrap state and not the last column.

[DHowett]

TL;DR: this is supported by standard behavior in either direction.

@j4james likely knows much more than I do about it. 🙂

[j4james]

We've tried to follow the spec as closely as possible (documented in STD 070, pages D-13 and D-14). I think the only intentional deviation is with the tab control, where the behavior was changed in later DEC terminals (in that case we're matching the most recent DEC implementation, rather than the older spec definition).

I believe this is exactly the same behavior as XTerm, and last I checked, MLTerm and the Linux console were also following the spec quite closely. We're definitely not the only terminal that handles EL this way. So if you're seeing something different in XTerm, it's probably worth investigating whether there is some other factor involved.

[DHowett]

Thanks!

And with that, I'll close this one as By Design.

[DHowett]

(Also, I think I need to install moar on all of my machines so thank you for that)

[o-sdn-o]

It seems to me that with such a design, a console application that outputs via a serial port and has no way to track the size of the terminal will inevitably periodically lose the last character. Moreover, with a terminal size of 1xN, the last character will always be lost.

[DHowett]

@o-sdn-o Literal serial terminals act like this, so.

[walles]

if you're seeing something different in XTerm

I haven't tried XTerm, but I did produce this list (before I just changed moar's behavior):

Test results for moar showing the last column

• ❌ Windows Terminal (on Windows)
• ❌ VSCode terminal on Windows. On macOS, the VSCode terminal works fine.
• ✅ VSCode terminal on macOS. On Windows, the VSCode terminal does not show the last column.
• ✅ iTerm2 on macOS
• ✅ macOS Terminal
• ✅ Hyper on macOS
• ✅ KiTTY on macOS
• ✅ WezTerm on macOS
• ✅ MobaXTerm on Windows: Wrapping long words eats up one character walles/moar#250 (comment)

I do appreciate the thorough response!

[o-sdn-o]

With this design, supporting the \e[K command is completely pointless. Applications that can track the terminal size can easily erase the rest of the line using spaces, and this will be even more efficient. Other applications cannot rely on this vt-command at all because of the risk of losing the last character.

[j4james]

@o-sdn-o It's worked this way for more than 40 years - I think it's a little late to decide you don't like the design. But if you want to invent a new protocol, maybe propose something on terminal-wg.