BASTERM v3 (now with RTS/CTS, port scan, and FindBaud!) [also C-code version]

[Xiphod]

I've never coded up flow control for a UART, so after tons of trial and error - I think I finally got it stable with BASTERM (a serial-terminal done using BASLOAD/BASIC for X16).

Whatever I did, I was able to transfer the entire script of Bee Movie in text file form (49KB) in a continuous stream without a single dropped character. The CTS/RTS lights on the little USB/serial adapter seem correct.

Now, it can't be (easily) helped that under BASIC, we can only stream at about 200 Bytes Per Seconds. That's just the price of a constantly interpreted runtime even at 8MHz. But with all the steps now figured out, I'll do a C version next and see if we can ramp up the performance. The point was, even if I up the serial connection to 115200 (in BASIC), I can still transfer the file without error. And I no longer have to play games inserting a delay between characters when sending text files.


The spec sheet wasn't clear to me at all in how to do this. Things got so bad, I even actually turned to ChatGPT! But it wasn't really any help either. After hours of reading and walking the history of the Bell 103 to the TR1402 on up to the 2651 and 8250, and learning about 16x up-sampling and "2 of 3" hits to find start bits, I finally came across a discussion on why the FIFO interrupt hits at 14 (i.e. "almost full") instead of 16. And turns out, the slow-ness of BASIC needs even a bit longer runway!

All I know is I've gone from having to artificially add 16ms delay between characters, to now 0ms delay and use whatever baud rate, and things go without any drops.


So this VERSION3 of BASLOAD will work with both the SERIAL and NETWORK "sides" of the TexElec serial card, but will require CTS/RTS be enabled (which is fairly universal these days, so not an issue). You can keyboard-TYPE stuff without flow control (as most keyboards can't even type much past 300 or 600 baud equivalent, in terms of overall data-rate), but any kind of sustained streaming really needs an assist.


I've also added two other neat features:

- a "port scanning" mode to find which serial ports you have (another trial and error process, but I think I got it figured out)

- a "baud rate finder" feature, so if you forget or just don't know what your device was left at (and it has no display to show it), this can automatically just try them in sequence and find what works.


Hope this helps as a starting reference for some on how to program the serial card. More to come, but this was a helpful step to getting things reliable (and even in BASIC!).

[Xiphod]

Couple reference images below:

1) Bee Movie script transferred in one continuous stream, all characters accounted for. This was done by connecting across a null-modem cable over to a USB/serial adapter on a modern-day i7-based laptop, then using ZOC from the PC to "Send a Text File" at whatever baud rate (I used 115200). Press F2 at the end of the transfer, and the received bytes count should match the size of the file that was sent.

2) And example of the "port scan" and "find baud" features. Not super exciting to look at, but just showing what they *should* look like. In NETWORK mode, the baud-finder is more automatic since it uses the ZiModem. In the SERIAL mode, whatever is on the other end of the cable will have to manually respond.

[Xiphod]

Also a note: the WiModem232 from CBMStuff now have a "null-modem mode" - it basically uses Bluetooth between two near-by WiModem232's for a high speed wireless pass-thru. And those WiModem232's have now unlocked the 460K and 921K speeds (I haven't gotten 921K to work reliably yet {will try new cables}, but 230K and 460K are good). Just AT*UPDATE to grab their new firmware.

Some wireless (WiFi) performance notes so far:

Code: Select all

Using 20MB test file (modern-PC to modern-PC, or ZOC to ZOC).

---------------------------- BELOW: (normal wifi modem @ about -50 dBm)
[ use ATDT<IP>:<port> to connect one WiModem232 to another on the LAN ]
ZModem @ 38400  -->  3.6KBps (many re-try errors but maintained decent rate)
ZModem @ 57600  -->  0.4KBps (too many errors, gave up; did not complete xfer)

YModem @ 57600  -->  1.9KBps
YModem @ 115200 -->  2.8KBps
YModem @ 230400 -->  3.7KBps
YModem @ 460800 -->  5.0KBps
(921K did not work; TBD with new cable)

---------------------------- BELOW: ("NULL-MODEM MODE" on WiModem, basically wireless-via-Bluetooth)
[ on older WiModem232 do AT*NULLMODEM0 ]
[ on newer WiModem232 Pro do AT*CABLE0 ]
YModem @ 115200 -->  6.7KBps
YModem @ 230400 --> 10.6KBps
YModem @ 460800 --> 13.0KBps
ZModem @ 460800 --> ~3.0KBps (while transfer worked, many re-try errors hindered the throughput)
(921K did not work; TBD with new cable)

And in some other testing (until we get a native data-streamer on X16), here is some wired serial-port performance:

Code: Select all

With wired null-modem cable between...
4.77MHz PC 5150 to Modern PC --> 0.9KBps (ZModem @ 9600 baud,  19200 and 38400 did work but at 0.6 and 0.8 KBps {excessive errors}, then did not work at all at 57600; conex 7.5 to ZOC)
286 to Modern PC --> 2.3 KBps   (ZModem @ 57.6K, faster was not reliable; conex 7.5 to ZOC; no errors reported, so unsure why this couldn't get over 5KBps; maybe FIFO throttled or need better cable, idk)
486 to Modern PC --> 11.3 KBps (ZModem @ 115.2K, conex 7.5 to ZOC; this needed a few error corrected retries, indicating on the edge of performance, i.e. a poor quality cable could reduce this)
Pentium to Modern PC --> 11.1 KBps (no retries, ZModem @ 115.2K under WinXP HT to ZOC; feels non-stressed)
Modern PC to Modern PC --> 18.9 KBps (no retries, YModem @ 460K, Windows ZOC to ZOC)
Modern PC to Modern PC --> 44.9 KBps (no retries, ZModem @ 460K, Windows ZOC to ZOC; state-of-the-art :) sort of, there are some 30MHz clocked UARTs out there by SIIG)
(not stable at 921K, unable to transfer files)

ZModem has some specific timing on ACKs that are difficult to maintain across any wireless connection, but it may be OK for wired connection. Still someday I'd like to re-test with YModem across these wired connections.
EDIT: In follow up testing, it seems accurate that YModem is better suited for wireless (variable speed connection), ZModem better suited for wired.

And yes, it's not the host system here (286, 486, Pentium) that controls the speed, but more about what specific UART they had and "baud generator". I was using laptops (yes even a 286 laptop) and so wasn't 100% sure. I suspect the 286/486 had an 8250 (which can do 115200 if the host system has enough horsepower). The Pentium was in an IBM ThinkPad and I can't find specs on what serial-parts it had (but being late 1990s, I'd hope a 16x50 of some sort, its WinXP driver capped at 128,000 baud).

NOTE on 286: I'm thinking the reason the '286 couldn't wire over >5KBps was maybe just the overall integration (12MHz system CPU, then the speed of its hard drive and RAM chips -- in that testing, it was tasked to write the received data to the file system after doing CRC checks on the blocks). Just speculating, but worried we might run into a similar wall on the X16.

[Xiphod]

My daughter happened to be around when I was testing, and suggested I try transferring The Bible across the wire! So, we did - we found a 5MB (megabyte) English version and gave it a go.

Was a good sustained test, and not a single character was dropped even when we set to 460Kbps connection (but it fell apart at 921Kbps, everything got misinterpreted -- but it's across about 16ft of cable). This gives me confidence that the CTS/RTS flow control is reliably working. This was using a C-version of what is fairly equivalent to the BASLOAD version (now attached below).

I couldn't get the find-baud feature working quite right, so that's disabled in this C version. Still an initial reference draft for testing purposes. One other difference is in this C version, I up'd the FIFO threshold from 4 to 8 - but it still didn't work at 14 (probably just overhead of the KERNAL and cc65 runtime and such). Note like most cc65 built things, this PRG you'll have to reboot to get out of - cc65 stuff doesn't gracefully return back to BASIC.

[Xiphod]

Some other testing: if the data is to be streamed to the display, there is some impact based on the screen size. This isn't surprising, just a reminder of how sensitive any background processing is to the performance (similar to video work, where I imagine dropped-characters in a serial stream being like "tearing" in a video presentation; i.e. indicator you're doing a bit too much processing in the foreground to sustain a desired performance in the background).

So far I don't think any implemented approach is really taking advantage of IER (interrupt enable register). Doing so means inlining a custom ISR to monitor for the interrupt line, then buffering data to service the interrupt (as opposed to just polling on if a new character has arrived). In theory the interrupt approach should let one work more in blocks of data more easily and improve overall efficiency.

Code: Select all

                      wired     wired     wireless   wired
                      CTERM     ROMTERM   ROMTERM    BASTERM
X16 SCREEN 0   80x60  1.8KBps
X16 SCREEN 1   80x30  2.8KBps   2.2KBps   2.0KBps    0.2KBps
X16 SCREEN 11  32x25  2.9KBps
X16 SCREEN 7   22x23  3.1KBps

By "wireless ROMTERM" I mean using the Zimodem software in the network/serial card, connecting via ATDT"<ip>:port" to another system on the LAN, then "ATA" to answer.

CTERM works a tad faster because it's raw (BSOUT) to the KERNAL and not managing its own status row.

[Xiphod]

Note on "methodology" of the test-setup. The reason for bothering with modernPC to modernPC is just verifying the achievable rates under the "best of conditions."

And correct, the setup here isn't the "best" of cable connections - just had to go with what I had, as far as cables and connections. But that'd be representative of a "typical casual setup." There are some "straight DB25/USB" cables (have some on the way), but caution that they might be null-modem cables if intended for something like a printer.


Being more of a purist, Jim went with the DB25 connection on the WiModem232 as the "OG" style of serial connection (very 1960's-ish but was also used in the 1977-1982-ish era of microcomputers {TRS-80 / NEC8001 have it, Poly88 used it for serial-keyboard, etc}; gradually the industry ditched those 2ndary serial pins no one was using, making the 9-pin connector become more prevalent starting around the mid-1980s - probably also as a practical matter, as the smaller 9-pin allowed more room for other connectors on the ISA back panel ). There are other WiModem type devices, but I like these from CBMStuff - he's had them for many years and the firmware is solid. Plus the "null modem emulation" feature that uses Bluetooth is very neat.


For "wired test" - it's the same setup, just a null-modem cable between the USB/serial adapters instead.

[Xiphod]

For the network/Zimodem side, I'm not sure if the RTS/CTS is working in the C version quite right (the "serial side" RTS/CTS seems ok). It's working for me in the BASIC version (both network and serial) and I'm not immediately seeing a difference.

Maybe it's ok (just seems to go awry after changing baud rate, again only in the C-code and Network use-case).
EDIT: Still seems to be an issue (in C version with the Zimodem/network interaction). Tested again the BASIC version and it seems fine.