Commander X16

VERSION 1.0 IS PUBLISHED


THIS IS A LINK TO THE 1.0 FINAL Specification
viewtopic.php?p=30617#p30617

Original post starts below:


A few day's ago I started a thread on DISCORD about standardizing a file format for Bitmap Images that is optimized for the Commander X16. There's about 1/2 dozen of us on the thread and we've made quite good progress over the past week. We are looking at publishing a version 1.0 of the specification by the first week of December. This project is open to constructive comments and suggestions from all interested parties.

Here is the DISCORD thread. https://discordapp.com/channels/5475596 ... 0908092416

I am posting Draft Specification Revision #4 here. Post comments and suggestion either here or on the DISCORD thread.


LIBRE OFFICE ODT FILE
MS-OFFICE DOC
ADOBE ACROBAT

My initial thoughts:

1. Why not make the version (byte 3) ASCII? It's not like you're going to need 256 versions. And being able to eyeball the file and see "BMX1" (or "BMX2" etc...) would be nice.
2. Why the 19 byte header padding? Why not make your header 16 bytes instead of 32 (with 3 bytes of padding)? 19 bytes just seems excessive. If you're adding some extension that needs that much space, you may as well bump the version number and be done with it.

I'm assuming the idea is that you want to be able to load images directly into VERA ram for immediate display... but uncompressed image data on disk just feels... wrong.
Given that you've got built-in LZSA2 decompression in the kernal (`memory_decompress` at $FEED) and that can decompress directly into VERA RAM, why not support that? Even if you're loading in off disk and have to load it into main RAM first... You've got a meg of banked ram to play with, so it seems like having lots of (compressed) images loaded would be a nice feature to support.
A complication would be that you'd want to be able to have images bigger than the 8KB bank size. I think kernal LOAD can load data into multiple banks in one go, but I don't think memory_decompress can deal with banking... but if the compression is done in 8KB boundaries in mind... hmm. Fiddly.

Similarly, seems wasteful always reserving room for a 256 entry palette. I'd recommend supporting smaller palettes (even if restricted to powers-of-two). I'm assuming the palette is designed to be directly usable by VERA too.

Its an interesting idea to make the version byte ASCII but one of the goals of this format is simplification for the developer. It would add an extra layer of complexity to decoding the file. (Admittedly minor)

EyeballKid wrote: ↑Mon Nov 20, 2023 4:14 am My initial thoughts:

2. Why the 19 byte header padding? Why not make your header 16 bytes instead of 32 (with 3 bytes of padding)? 19 bytes just seems excessive. If you're adding some extension that needs that much space, you may as well bump the version number and be done with it.

So it's on a word boundary and who knows what might be needed in the future.

ALSO the one "mainstream" graphics format that has changed it's header size is BMP. It causes existing programs to break when you do this. I use PSP 3.11 on an XP box at home and in a Windows XP VM. It's over 20 years old. If I export a JPG or PNG from GIMP, then PSP can still open it. It chokes hard on modern BMP's

I'm assuming the idea is that you want to be able to load images directly into VERA ram for immediate display... but uncompressed image data on disk just feels... wrong.
Given that you've got built-in LZSA2 decompression in the kernal (`memory_decompress` at $FEED) and that can decompress directly into VERA RAM, why not support that? Even if you're loading in off disk and have to load it into main RAM first... You've got a meg of banked ram to play with, so it seems like having lots of (compressed) images loaded would be a nice feature to support.
A complication would be that you'd want to be able to have images bigger than the 8KB bank size. I think kernal LOAD can load data into multiple banks in one go, but I don't think memory_decompress can deal with banking... but if the compression is done in 8KB boundaries in mind... hmm. Fiddly.

Similarly, seems wasteful always reserving room for a 256 entry palette. I'd recommend supporting smaller palettes (even if restricted to powers-of-two). I'm assuming the palette is designed to be directly usable by VERA too.

The palette can be directly loaded to $1FA00 in VERA Ram.

I kind of felt this way myself at 1st but these days disk space is so cheap. The maximum practical size of uncompressed image data for this specification is 76.8 kilobytes which is a full screen of 320x240 in 8bpp mode. It's not the specification definition because it is left to the programmer to handle larger images.

Compression wouldn't gain us very much really. But I am considering RLE and LZSA compression in a future rev. Could easily drop that RESERVED section to 18 bytes.


Thank you very much for taking the time to comment.

ahenry3068 wrote: ↑Mon Nov 20, 2023 6:52 am So it's on a word boundary and who knows what might be needed in the future.

Is there something significant about 32 that makes it a better boundary than 16?

The palette can be directly loaded to $1FA00 in VERA Ram.

Oh, yeah, I'd forgotten that palette was hardwired at $1FA000... so if you load the whole image in one go (kernal LOAD), you've still got to copy the palette data into place So I still think supporting a smaller palette would be worthwhile (it's not really any harder copying a subset of a palette rather than a full one - and I can actually think of some obscure but fun reasons why you might want images which use different subsets of the palette...).

I kind of felt this way myself at 1st but these days disk space is so cheap. The maximum practical size of uncompressed image data for this specification is 76.8 kilobytes which is a full screen of 320x240 in 8bpp mode. It's not the specification definition because it is left to the programmer to handle larger images.

Fair enough. How about the use case where a program wants to load in multiple images (either into main RAM or banked RAM). In that case the space _is_ a big consideration. And kernal memory_decompress _does_ support decompressing directly into VERA RAM, so it's nice and simple to get the image data up onto the screen...

I'm not really arguing for any changes - I know you've all thought about this a lot more than I have. Just wanted to kick the tires

EyeballKid wrote: ↑Mon Nov 20, 2023 7:27 am

ahenry3068 wrote: ↑Mon Nov 20, 2023 6:52 am So it's on a word boundary and who knows what might be needed in the future.

Is there something significant about 32 that makes it a better boundary than 16?

The palette can be directly loaded to $1FA00 in VERA Ram.

Oh, yeah, I'd forgotten that palette was hardwired at $1FA000... so if you load the whole image in one go (kernal LOAD), you've still got to copy the palette data into place So I still think supporting a smaller palette would be worthwhile (it's not really any harder copying a subset of a palette rather than a full one - and I can actually think of some obscure but fun reasons why you might want images which use different subsets of the palette...).

I kind of felt this way myself at 1st but these days disk space is so cheap. The maximum practical size of uncompressed image data for this specification is 76.8 kilobytes which is a full screen of 320x240 in 8bpp mode. It's not the specification definition because it is left to the programmer to handle larger images.

Fair enough. How about the use case where a program wants to load in multiple images (either into main RAM or banked RAM). In that case the space _is_ a big consideration. And kernal memory_decompress _does_ support decompressing directly into VERA RAM, so it's nice and simple to get the image data up onto the screen...

I'm not really arguing for any changes - I know you've all thought about this a lot more than I have. Just wanted to kick the tires

32 vs 16. Not really but I wanted to allocate a little extra room for expansion later, because who knows .

I allowed for loading a partial Palette with the significant Palette entry's field. The Palette section being a fixed size allows for simpler code when reading the file. Everything that is known before hand and doesn't have to be calculated at run time makes the programmers life just a little simpler.

My first thought was that the data should be in an order that made it possible to read an image straight to VRAM without having to copy anything afterwards. Unfortunately, the memory layout will not allow this. Even if you relocate the textlayer, there’s still the PSG registers between the available video memory and the palette. So regardless of image format you will always either have to copy some data or divide the image in two files. It’s a pity, but there are surely other reasons for the chosen memory layout.

In short, I think it looks good, I am looking forward to this. Good work!

EyeballKid wrote: ↑Mon Nov 20, 2023 4:14 am 1. Why not make the version (byte 3) ASCII? It's not like you're going to need 256 versions. And being able to eyeball the file and see "BMX1" (or "BMX2" etc...) would be nice.

I get that! And it would still be easy to code with it. But it would limit us to 9 versions... After that we can still introduce sub-versions somewhere else in the header.

EyeballKid wrote: ↑Mon Nov 20, 2023 4:14 am I'm assuming the idea is that you want to be able to load images directly into VERA ram for immediate display... but uncompressed image data on disk just feels... wrong.

I had the same feeling, but then I tried to calculate the tradeoff/benefit of compressed files in regards to loading time vs. decompression time: according to my calculations we would not benefit in regards to total loading time with compressed files. At least not for SD cards. With an actual disc drive, the benefit would be dramatic. But who uses those for real?

I actually designed a universal binary file container a few years ago, with the intent that it could contain any media type, including bitmap files.

It might be worth a look.

The basic idea is that the file is broken into 3 block types: the header, an index, and the data

The header is just basically name and creator info, which is entirely discarded by the loader.

The index tells the loader where to start to read each block. It contains the block type, position, and length in the data file.

The data blocks are just binary data.

With this, you don't need a special data format for images; you just load the data blocks in the right locations in VERA, then set the video mode accordingly. (Ideally, by including something in the Image Metadata block to say "Use this video mode.")

Here's the spec: https://docs.google.com/document/d/1v8o ... sp=sharing

This is still a draft. There was a lot of discussion on the DISCORD thread over the past couple days. This is REV 5 of the format.

@TomXP411. Desertfish had a similiar idea without the container header. What he suggested was almost a Video Tracker file for the X16.
That may be a great direction to go for some programs. But part of what I wanted to do with this was also make "Not full Screen" bitmaps possible. This format is meant for things like Paint Programs and also Illustrations and Bitmaps for Games or other software, (Maybe even embedded images in a Word Processing file, if anyone ever want's to write something like that.) There are a few significant changes in this one. But I'm optimistic that we can publish a specification by Week 1 in December.


ADOBE ACROBAT
LIBRE OFFICE
MICROSOFT OFFICE

Very minor revision. Should break almost no code written to Rev 5.
Expect to call this BMX Specification version 1.0 as of 1 December 2023 unless something major comes up with it.

ADOBE ACROBAT
LIBRE OFFICE
MICROSOFT WORD