Remove most VT52, VT100-VT525, ANSI escape sequences

[mobluse]

It would be good if all escape sequences could be removed when printing to the screen, but not removed when echoing to a serial terminal or the console the emulator runs in, since terminal escape sequences that are not supported should be ignored by the device. This would also make it possible to run more programs that use e.g. ANSI escape sequences without modification and a connected terminal could act on those sequences. As it is now Escape is ignored, but not the characters after it, when printing to the screen.

The command sed 's/\x1b[[(]*[ ?!,0-9;]*[\x0-\x1a=>@-Za-z~]//g;s/\x1b.*[\\\x7\$]//g' removes most terminal escape sequences and doesn't remove real text except in these cases:

ESC 7 Save Cursor Position in Memory DECSC
ESC 8 Restore Cursor Position from Memory DECSR 
ESC ( 0 Designate Character Set – DEC Line Drawing
ESC Y rc VT52 Direct Cursor Addressing
There might be other more obscure Escape codes that are also not terminated correctly, but since a control character or letter is usually printed often, not many characters would be ignored.

ESC 7 and ESC 8 can sometimes be replaced with ESC [ s and ESC [ u.

For this no real text is removed but the two visible characters for the coordinates are printed on the screen:
ESC Y rc VT52 Direct Cursor Addressing

If X16, in the outputed characters, encounters the Escape character it should not print it to screen and ignore the following characters until after it encounters a character with a code less than 27 or greater than 63. This is a simple rule because an escape sequence can contain ESC (27) and ? (63), but it can be ended with e.g. BEL (7) or a character equal to or greater than @ (64). This is a simpler rule than the sed command above.

I think this could be implemented so that printing is hardly slowed down. It is similar to quote mode. It's also better to start implementing correct Escape handling now so that people don't depend on the non standard behavior after Escape that exists now.

[kelli217]

The X16 screen editor is not an ANSI terminal.

And the -echo output from the emulator is just a data dump, pure and simple. If it should happen to contain a valid ANSI escape sequence, that's just coincidence.

However, if someone were to write a terminal program for the system, then your recommendations certainly make sense.

[mobluse]

X16 has a PETSCII terminal built-in and I have never thought it had an ANSI terminal. The reason for my suggestion is that you could connect a serial terminal to the real X16 or the emulator and that would probably have an ANSI terminal. You could make programs that work well with the PETSCII terminal and also uses the features of the ANSI terminal, if my suggestion was implemented. Also it is rather simple to implement and doesn't slow down printing since it only affects ESC and the following characters.

I have developed this system https://github.com/mobluse/x16-petscii2utf8 to convert PETSCII to UTF-8 and ANSI escape codes. One could use more features of e.g. the VT525 terminal if one could print escape codes and they would be ignored on the X16 screen.

[kelli217]

Yeah... that would be someone writing a terminal program for the machine.

Something like the old COMM TERM, or Common Sense, or DESterm, or NovaTerm, etc.

Or something that explicitly supports Commodore/PETSCII graphics, such as CCGMS.

[mobluse]

Well, I'm talking about the opposite, i.e. connecting a terminal to X16; not using X16 as a terminal. My suggestion requires changes to the print routines in ROM. To write a terminal program you don't need to change the ROM.

[DragWx]

This may already be possible by overriding the Kernal's IBASIN and IBSOUT vectors with custom handlers that can convert between ANSI and PETSCII. In this case, this would be a separate program you load and run (like a DOS TSR program) before you load and run your ANSI application.

[kelli217]

Oh. I see. So, something like the way CP/M computers operated back in the day; they were controlled through a serial terminal, and the later models that had their screens and keyboards built in still used a terminal-style I/O for the local console (I seem to recall that the 128's CP/M mode had the keyboard and screen presented to the system as though it were an ADM-3A terminal, or the more advanced ADM-31).

Considering how PETSCII-focused the X16's BASIC is, it'd probably be better to use something like CGTerm (available on modern platforms) to access the system if you wanted to take that approach. You'd also have to redirect the channel I/O systems on the X16 side so that device 0 and 3 would go through the serial connection.