Commander X16

Some of you may have seen my other posts about my attempts to port the game "Another World" from the Amiga to the commander X16. I'll use this thread to keep you all updated on my progress.

What started out as a learning exercise soon became a full on project. I originally coded it in C using CC65, but found that it ran very slow.

I scrapped my C code and taught myself 65c02 assembly, and re-wrote the entire thing in pure assembly. Along the way I have learned a lot about the 65c02, the CX16, the Kernal, and the VERA.

The game is rendered using 2d polygonal vector graphics. The file formats are quite unique, you can tell that it was designed by one person with little outside influence. One of the awesome things was that the polygons are defined in a way that suits the polygon fill helper in the VERA perfectly.

Using the VERA FX Polygon Fill Helper

I set out to write a filling algorithm using the VERA FX. One of the most challenging things about this port is the fact that the Amiga is a 16-bit computer, and the CX16 only 8. The 320 pixel width has caused me so many headaches. Writing a signed 24-bit division routine for the polygon fill helper wasn't fun either(you need it for slope calcs). I had to learn a lot about how division and multiplication works on an 8-bit computer. Anyway, I finally got it working, but it turns out that the game code uses a lot of dodgy "hacks" to clip or not clip pixels. So you need to emulate that perfectly or else your polygons don't display correctly (or at all). The polygon fill helper is somewhat rigid in its requirements, so for things like clipping and overflowing it gets very complicated fast. I got it about 75% working, but it wasn't acceptable so I took a different approach.

Alternative polygon filling

It turns out the Commander X16 is pretty fast anyway. I wrote scanline table filler. I'm not sure if it has a name, it's my own design. The polygons in the game are all fortunately limited to 256x256 in size (they are scaled later), so I keep a left and a right side table and using bresenham I "draw" lines between vertices in for each side of the polygon. Then using those tables, I scan down them one line at a time and draw filled horizontal lines between the sides. It results in a perfect replica of the original polygon.

My horizontal line algorithm uses the 32-bit cache writes of the VERA FX. The polygon routines I have now are about 90% correct. There still seems to be a couple of edge cases where a polygon isn't drawn, or it overflows to the next line.

Here is a real-time gif of the game intro: Game Speed

The original game actually ran at 50hz. I was hoping to get at least 20fps for the game, but to my surprise my code was fast enough to match the original speed, in fact it can run faster! At the moment I haven't implemented any interrupts I am using the kernal default, which is 60hz, but it still looks great.

I was so worried about speed that I decided to make my screen resolution 256x192. However since the speed is so good I'm going to go to 320x192. I will have to update all the X values to 16-bit which is painful.

VRAM

Why 320x192 and not the original 320x200? Well this is because of the way the VERA addresses the VRAM in blocks of 2048 bytes. If I went to the next block, it would have to be 320x204. If I used that resolution, I wouldn't have enough VRAM for the (triple) buffering the game uses. That said I may revisit this scenario, I might be able to make it work, albeit with a 4 pixel black bar at the bottom of the screen, and I'd have to sacrifice the text mode font. Other system implementations (eg MS-DOS) use a screen buffer for display, but because of the way the VERA handles page switching, I don't need that and have devised a way to rid the game of slow screen copies every frame (not completely unavoidable, but a whole page less every frame). You can see more about how the screen buffering works on this breakdown: https://fabiensanglard.net/another_worl ... index.html.

Next Steps

I've got about 80% of the game engine opcodes working. I have rudimentary keyboard support. The main thing I'm aiming for is to get it to play the entire introduction and first level without any problems. At the moment, the intro finishes flaky, and the first level doesn't load. That said, it's likely something small. The game runs entirely on a virtual machine so if most of those things work then the rest will too. I'm currently working on changing the screen res from 256 to 320. The fact that everything is 16-bit is very draining.

Previously I also have said sound isn't possible, but I may have worked out a way to do it. The main limitation is the 512kb of ram. But since loading from the SD card is almost as fast as loading from memory I might be able to stream sound from the disk instead of memory. Alternatively, I could just make it a "feature" if you have 2mb of RAM. It would also depend if it could be done fast enough but I'm confident it could be. Also I've extracted all the resources manually, I need to implement that in the game.

Still to do really are:
- Get game going at original screen res 320x200
- Fix any broken implementations of opcodes
- Finish keyboard input and implement gamepad input
- Text rendering (should I use default font?)
- Implementation of resource decompression
- Sound
- Optimisations (there are heaps)
- Full playthrough testing

I've been toying with the idea of making a few videos on my work for youtube. I'm not a youtuber or anything but I know it would be something I'd watch haha. There's heaps I haven't mentioned here. I've written several debugging tools, disassemblers for the game, resource extractors, and other stuff. Let me know if you'd be interested.

This is awesome. Last year I actually E-mailed the game's original developer hoping to interest him in doing a Port. (No reply)...

This was one of my favorite games from the 90's... (I first played it on MS-DOS)..

I for one would be very interested in seeing any Youtube videos of your work...

Is the original Audio PCM... Streaming should be a valid option for the sound. A new faster read mode was introduced in ROM r47. You can realistically expect about 150000 bytes / sec from disc (Though that doesn't account for doing any other processing). I've achieved over 200000 bytes per second on some SD-Cards but there's a significant variation between different SD-Cards. If I was targeting a game I wouldn't count on > 150000 bytes / sec.

Not sure, if I'm saying something dumb here, but you could use the 200px and overlay the last line with the second layer and a line filled with an eight by eight character, which is transparent on the first four lines and all black on the second four lines. With that, you can easily reuse the part below the masked four lines. If that helps in any way, just a late night, definitely need to go to bed now idea...

I was so worried about speed that I decided to make my screen resolution 256x192. However since the speed is so good I'm going to go to 320x192. I will have to update all the X values to 16-bit which is painful.

You can mitigate the pain a little if you remember that the Upper word can't ever be greater than 1 (unless your using signed or off screen coordinates). That can help short circuit the 16 bit math a little. I've dealt with the same issue for the same reason when I was developing my BMX loading and BitBlt routines. ( You never have to do an actual add or subtract instruction to the Upper Word, it's either "On" or "Off" )

Ok I think I've worked out the 320x200 mode, as it turns out I had extra VRAM to spare after all. All the ports I've seen of the game seem to use 4 screen buffers, but I think one is the display, so in essence you only need 3, which means I have enough memory to run the game at 320x200.

Here's the higher res version (in real-time): There is still a few minor drawing issues, but it's mostly there. The game intro gets up to the part where he sets the experiment going, but then crashes. I've written another program that disassembles the bytecode so I've been using that to debug where I might have incorrect opcode implementations and I'll also try to see if it goes beyond 3 back buffers - to my knowledge it has a display buffer, a draw buffer and a background buffer. The opcodes for page switching are weird... it has a direct and indirect mode, and uses page pointers, so it's difficult to identify page switching problems.

Some of the conditional opcodes can be tricky, also all it takes is one wrong read or not considering a high byte or signedness somewhere and it all falls down.

Is the original Audio PCM...

I believe it's a form of 8-bit PCM encoded audio... I'm not a sound guy I'm more graphics and I'm yet to look into it very deeply. I'll need to experiment a bit first I think to see if I can stream it. It also needs to be decompressed too, which could be quite taxing on the CPU. The only way may be to decompress to a file first on game startup or something like that. My goal is to make the experience as intuitive as possible - ie the user doesn't need to do any preparation, they just drop the game files in and it works.

you could use the 200px and overlay the last line with the second layer and a line filled with an eight by eight character, which is transparent on the first four lines and all black on the second four lines.

You're saying use a text layer to mask the bottom of the screen? Not such a bad idea. So far I think I've got it working good for now though. I will probably use the second layer for in game text which I haven't implemented yet.

if you remember that the Upper word can't ever be greater than 1 (unless your using signed or off screen coordinates)

Unfortunately the game does use signed values - 16 bit. I have used the trick on a few occasions though to test if the value is outside of 320 - ie the value in the high byte will be greater than 1 for early branching. Example: The other thing worth mentioning is that the game uses a special trick to squeeze a bit of extra space out of storing the X and Y values in 2 bytes: if the Y value is higher than 200, then the difference gets added to X.

Example:
X = 255 ($FF)
Y = 250 ($FA)

if Y>200, Y=Y-200:
Y= 250-200
X = X + (250-200)

X = 305
Y = 200

This gives you X values in the range of 0 - (255 + 55), or 0-310.

I believe it's a form of 8-bit PCM encoded audio... I'm not a sound guy I'm more graphics and I'm yet to look into it very deeply. I'll need to experiment a bit first I think to see if I can stream it. It also needs to be decompressed too, which could be quite taxing on the CPU. The only way may be to decompress to a file first on game startup or something like that. My goal is to make the experience as intuitive as possible - ie the user doesn't need to do any preparation, they just drop the game files in and it works.

I'm super stoked for this particular game actually. I don't have the chops you do for the whole game and I'm glad your doing it. But perhaps with this one little thing I could help. I've been doing a lot of looking at Digital sound and the X16 the last few months with developing a WAV player and a Mono Streaming video routine with a sound track. I don't currently have a copy of the game so could you please forward me a short (at least 10 or may 30 seconds) sample from the Game's original Audio I'll see if I can figure out the format and a decoding routine for you.

If it turns out to be ADPCM decoding the Audio on the fly probably isn't doable (at least with all the other processing in concert) but creating a new file on initial game startup should be doable pretty easily.


*** NEVER MIND ABOUT SENDING ME A SAMPLE, LOOKING AT THIS A BIT I REALIZE THAT'S A BIG ASK. I'M LOOKING AT THIS MYSELF RIGHT NOW ***

I realize this may be too late in the cycle for this game, but it might be of use if you go on to try to port Flashback or Heart of the Alien, but perhaps there could be a way to selectively implement the FX Triangle Rasterization mode. What I mean is invoking it for areas when/where clipping is not needed by the engine, switching it off when it's called for, and then switching it back on until the engine requires clipping. Or would the branching resulting be too unpredictable, and sap too many clock cycles, resulting in too uneven a framerate?

As for audio, for the music sensu-strictu, perhaps a rewrite to YM2151 and VERA geometry synthesis channels is called for, leaving the PCM channel for percussion that won't work with FM instruments or simple periodic noise, and in-game sound effects? Did the SoundBlaster track on the MS-DOS version include notes for the optional SAA1099s? It might not necessarily sound quite as good overall as full PCM for everything, but using the YM3812 FM track as a basis might lower the audio budget enough while still offering a leg up over hypothetical Commodore 64, Atari 8-bit, or Apple II or BBC Micro + Mockingboard versions that it could offer a plausible coul-have-been for a port an release from back in the day.

Then again, all of my ideas could cause more problems than they solve.

I have the Apple IIGS port (which is in turn a port of the SNES version, both were done by the amazing Becky Heineman) and took a look at it, but it just has one giant file called OOTWSOUND (which takes up all of the second disk, LOL) which has no data fork, only a resource fork containing 85 items with a custom resource type. I'd extract them and try to figure out how they're compressed, but I'd bet the files from the original Amiga version are in some more standard format.

perhaps there could be a way to selectively implement the FX Triangle Rasterization mode

Not a bad idea. The FX Rasterisation is about 10-20% faster than my current algorithm, so I'd have to test if there is much improvement through the additional overhead when selecting the method. I've optimised my method quite a lot, the frame rate is about 15-20fps which is playable, but I'd love to make it faster. I'll definitely look into it.

I've studied the DOS version audio a bit more (which is the data I'm using), the samples are Signed 8-bit PCM 11khz mono. The game has opcodes for sfx and also for a simplified mod-tracker style music. Should be able to play pretty fast I'd expect, I think I only need to pipe in about 200 bytes per interrupt at most if my maths is correct which would lose me only about 500-1000 cycles at most per 60hz.

the Apple IIGS port

I considered looking at it and the SNES version too, but the architecture is still quite different (especially the SNES) when it comes to optimisation and graphics, and given that both versions are 16-bit I decided they're probably not too helpful.

My version might actually be the first 8-bit version! (Although I saw a C64 version someone posted on Youtube years ago - not sure if it's real).

I'm getting really close to having something fully playable now, I will release the game for free (minus the assets) in the next few months. I don't get a lot of time to work on it unfortunately.

The only things not working now are bugs. I really appreciate the talent of programmers back in the day, it kills me to not have any decent modern debugging tools. I'm resigned to alternating between the VERA monitor on box16, or the debugger in x16emu. I waste a lot of time trying to find bugs.
I also discovered recently the game does use 4 back buffers, not 3 as I originally thought. I still have just enough VRAM, but I lose about 8 pixels at the bottom and also Tile Mode for text, so I'll be implementing that separately. I'll try to chase down the original Amiga font for authenticity

Still outstanding:
- re-writing and merging my sound tests into the main code
- write a music player
- fixing opcode bugs - I think it's in the conditional stuff
- polygon scaling bug - my fixed point maths appears correct, but it's not scaling correctly.
- intialisation code - the game needs to decompress the data files before using them
- finalise input - keyboard, and joystick.

As I learn new things about the CX16 and 65c02 assembly, I've been putting it into the game. Things like macros, and jump tables etc. I wrote a macro that will let me set the game resolution to test out different speeds. Very handy.

It's been a really fun exercise so far and I've learned a lot! I also have another original project I've started for the X16 which should be a unique game for the platform I hope.
Here is an obligatory GIF update - note the improved resolution, the graphics are probably 95% accurate now. There are a few artifacts etc but those are bugs in opcodes, not the drawing routines. It's a lot faster now too if you compare it to the previous ones.