VERSION 1.0 IS PUBLISHED
THIS IS A LINK TO THE 1.0 FINAL Specification
viewtopic.php?p=30617#p30617
Original post starts below:
A few day's ago I started a thread on DISCORD about standardizing a file format for Bitmap Images that is optimized for the Commander X16. There's about 1/2 dozen of us on the thread and we've made quite good progress over the past week. We are looking at publishing a version 1.0 of the specification by the first week of December. This project is open to constructive comments and suggestions from all interested parties.
Here is the DISCORD thread. https://discordapp.com/channels/5475596 ... 0908092416
I am posting Draft Specification Revision #4 here. Post comments and suggestion either here or on the DISCORD thread.
LIBRE OFFICE ODT FILE
MS-OFFICE DOC
ADOBE ACROBAT
A BITMAP FILE SPECIFICATION FOR THE X16. (1.0 is published)
- ahenry3068
- Posts: 1131
- Joined: Tue Apr 04, 2023 9:57 pm
A BITMAP FILE SPECIFICATION FOR THE X16. (1.0 is published)
Last edited by ahenry3068 on Sat Apr 20, 2024 3:26 pm, edited 2 times in total.
-
- Posts: 18
- Joined: Sat Jul 29, 2023 8:29 am
Re: A BITMAP FILE SPECIFICATION FOR THE X16. (In progress)
My initial thoughts:
1. Why not make the version (byte 3) ASCII? It's not like you're going to need 256 versions. And being able to eyeball the file and see "BMX1" (or "BMX2" etc...) would be nice.
2. Why the 19 byte header padding? Why not make your header 16 bytes instead of 32 (with 3 bytes of padding)? 19 bytes just seems excessive. If you're adding some extension that needs that much space, you may as well bump the version number and be done with it.
I'm assuming the idea is that you want to be able to load images directly into VERA ram for immediate display... but uncompressed image data on disk just feels... wrong.
Given that you've got built-in LZSA2 decompression in the kernal (`memory_decompress` at $FEED) and that can decompress directly into VERA RAM, why not support that? Even if you're loading in off disk and have to load it into main RAM first... You've got a meg of banked ram to play with, so it seems like having lots of (compressed) images loaded would be a nice feature to support.
A complication would be that you'd want to be able to have images bigger than the 8KB bank size. I think kernal LOAD can load data into multiple banks in one go, but I don't think memory_decompress can deal with banking... but if the compression is done in 8KB boundaries in mind... hmm. Fiddly.
Similarly, seems wasteful always reserving room for a 256 entry palette. I'd recommend supporting smaller palettes (even if restricted to powers-of-two). I'm assuming the palette is designed to be directly usable by VERA too.
1. Why not make the version (byte 3) ASCII? It's not like you're going to need 256 versions. And being able to eyeball the file and see "BMX1" (or "BMX2" etc...) would be nice.
2. Why the 19 byte header padding? Why not make your header 16 bytes instead of 32 (with 3 bytes of padding)? 19 bytes just seems excessive. If you're adding some extension that needs that much space, you may as well bump the version number and be done with it.
I'm assuming the idea is that you want to be able to load images directly into VERA ram for immediate display... but uncompressed image data on disk just feels... wrong.
Given that you've got built-in LZSA2 decompression in the kernal (`memory_decompress` at $FEED) and that can decompress directly into VERA RAM, why not support that? Even if you're loading in off disk and have to load it into main RAM first... You've got a meg of banked ram to play with, so it seems like having lots of (compressed) images loaded would be a nice feature to support.
A complication would be that you'd want to be able to have images bigger than the 8KB bank size. I think kernal LOAD can load data into multiple banks in one go, but I don't think memory_decompress can deal with banking... but if the compression is done in 8KB boundaries in mind... hmm. Fiddly.
Similarly, seems wasteful always reserving room for a 256 entry palette. I'd recommend supporting smaller palettes (even if restricted to powers-of-two). I'm assuming the palette is designed to be directly usable by VERA too.
- ahenry3068
- Posts: 1131
- Joined: Tue Apr 04, 2023 9:57 pm
Re: A BITMAP FILE SPECIFICATION FOR THE X16. (In progress)
Its an interesting idea to make the version byte ASCII but one of the goals of this format is simplification for the developer. It would add an extra layer of complexity to decoding the file. (Admittedly minor)
ALSO the one "mainstream" graphics format that has changed it's header size is BMP. It causes existing programs to break when you do this. I use PSP 3.11 on an XP box at home and in a Windows XP VM. It's over 20 years old. If I export a JPG or PNG from GIMP, then PSP can still open it. It chokes hard on modern BMP's
I kind of felt this way myself at 1st but these days disk space is so cheap. The maximum practical size of uncompressed image data for this specification is 76.8 kilobytes which is a full screen of 320x240 in 8bpp mode. It's not the specification definition because it is left to the programmer to handle larger images.
Compression wouldn't gain us very much really. But I am considering RLE and LZSA compression in a future rev. Could easily drop that RESERVED section to 18 bytes.
Thank you very much for taking the time to comment.
So it's on a word boundary and who knows what might be needed in the future.EyeballKid wrote: ↑Mon Nov 20, 2023 4:14 am My initial thoughts:
2. Why the 19 byte header padding? Why not make your header 16 bytes instead of 32 (with 3 bytes of padding)? 19 bytes just seems excessive. If you're adding some extension that needs that much space, you may as well bump the version number and be done with it.
ALSO the one "mainstream" graphics format that has changed it's header size is BMP. It causes existing programs to break when you do this. I use PSP 3.11 on an XP box at home and in a Windows XP VM. It's over 20 years old. If I export a JPG or PNG from GIMP, then PSP can still open it. It chokes hard on modern BMP's
The palette can be directly loaded to $1FA00 in VERA Ram.I'm assuming the idea is that you want to be able to load images directly into VERA ram for immediate display... but uncompressed image data on disk just feels... wrong.
Given that you've got built-in LZSA2 decompression in the kernal (`memory_decompress` at $FEED) and that can decompress directly into VERA RAM, why not support that? Even if you're loading in off disk and have to load it into main RAM first... You've got a meg of banked ram to play with, so it seems like having lots of (compressed) images loaded would be a nice feature to support.
A complication would be that you'd want to be able to have images bigger than the 8KB bank size. I think kernal LOAD can load data into multiple banks in one go, but I don't think memory_decompress can deal with banking... but if the compression is done in 8KB boundaries in mind... hmm. Fiddly.
Similarly, seems wasteful always reserving room for a 256 entry palette. I'd recommend supporting smaller palettes (even if restricted to powers-of-two). I'm assuming the palette is designed to be directly usable by VERA too.
I kind of felt this way myself at 1st but these days disk space is so cheap. The maximum practical size of uncompressed image data for this specification is 76.8 kilobytes which is a full screen of 320x240 in 8bpp mode. It's not the specification definition because it is left to the programmer to handle larger images.
Compression wouldn't gain us very much really. But I am considering RLE and LZSA compression in a future rev. Could easily drop that RESERVED section to 18 bytes.
Thank you very much for taking the time to comment.
-
- Posts: 18
- Joined: Sat Jul 29, 2023 8:29 am
Re: A BITMAP FILE SPECIFICATION FOR THE X16. (In progress)
Is there something significant about 32 that makes it a better boundary than 16?ahenry3068 wrote: ↑Mon Nov 20, 2023 6:52 am So it's on a word boundary and who knows what might be needed in the future.
Oh, yeah, I'd forgotten that palette was hardwired at $1FA000... so if you load the whole image in one go (kernal LOAD), you've still got to copy the palette data into place So I still think supporting a smaller palette would be worthwhile (it's not really any harder copying a subset of a palette rather than a full one - and I can actually think of some obscure but fun reasons why you might want images which use different subsets of the palette...).The palette can be directly loaded to $1FA00 in VERA Ram.
Fair enough. How about the use case where a program wants to load in multiple images (either into main RAM or banked RAM). In that case the space _is_ a big consideration. And kernal memory_decompress _does_ support decompressing directly into VERA RAM, so it's nice and simple to get the image data up onto the screen...I kind of felt this way myself at 1st but these days disk space is so cheap. The maximum practical size of uncompressed image data for this specification is 76.8 kilobytes which is a full screen of 320x240 in 8bpp mode. It's not the specification definition because it is left to the programmer to handle larger images.
I'm not really arguing for any changes - I know you've all thought about this a lot more than I have. Just wanted to kick the tires
- ahenry3068
- Posts: 1131
- Joined: Tue Apr 04, 2023 9:57 pm
Re: A BITMAP FILE SPECIFICATION FOR THE X16. (In progress)
EyeballKid wrote: ↑Mon Nov 20, 2023 7:27 amIs there something significant about 32 that makes it a better boundary than 16?ahenry3068 wrote: ↑Mon Nov 20, 2023 6:52 am So it's on a word boundary and who knows what might be needed in the future.
Oh, yeah, I'd forgotten that palette was hardwired at $1FA000... so if you load the whole image in one go (kernal LOAD), you've still got to copy the palette data into place So I still think supporting a smaller palette would be worthwhile (it's not really any harder copying a subset of a palette rather than a full one - and I can actually think of some obscure but fun reasons why you might want images which use different subsets of the palette...).The palette can be directly loaded to $1FA00 in VERA Ram.
Fair enough. How about the use case where a program wants to load in multiple images (either into main RAM or banked RAM). In that case the space _is_ a big consideration. And kernal memory_decompress _does_ support decompressing directly into VERA RAM, so it's nice and simple to get the image data up onto the screen...I kind of felt this way myself at 1st but these days disk space is so cheap. The maximum practical size of uncompressed image data for this specification is 76.8 kilobytes which is a full screen of 320x240 in 8bpp mode. It's not the specification definition because it is left to the programmer to handle larger images.
I'm not really arguing for any changes - I know you've all thought about this a lot more than I have. Just wanted to kick the tires
32 vs 16. Not really but I wanted to allocate a little extra room for expansion later, because who knows .
I allowed for loading a partial Palette with the significant Palette entry's field. The Palette section being a fixed size allows for simpler code when reading the file. Everything that is known before hand and doesn't have to be calculated at run time makes the programmers life just a little simpler.
-
- Posts: 292
- Joined: Wed Jun 03, 2020 11:33 am
- Location: Kalmar, Sweden
Re: A BITMAP FILE SPECIFICATION FOR THE X16. (In progress)
My first thought was that the data should be in an order that made it possible to read an image straight to VRAM without having to copy anything afterwards. Unfortunately, the memory layout will not allow this. Even if you relocate the textlayer, there’s still the PSG registers between the available video memory and the palette. So regardless of image format you will always either have to copy some data or divide the image in two files. It’s a pity, but there are surely other reasons for the chosen memory layout.
In short, I think it looks good, I am looking forward to this. Good work!
In short, I think it looks good, I am looking forward to this. Good work!
Re: A BITMAP FILE SPECIFICATION FOR THE X16. (In progress)
I get that! And it would still be easy to code with it. But it would limit us to 9 versions... After that we can still introduce sub-versions somewhere else in the header.EyeballKid wrote: ↑Mon Nov 20, 2023 4:14 am 1. Why not make the version (byte 3) ASCII? It's not like you're going to need 256 versions. And being able to eyeball the file and see "BMX1" (or "BMX2" etc...) would be nice.
I had the same feeling, but then I tried to calculate the tradeoff/benefit of compressed files in regards to loading time vs. decompression time: according to my calculations we would not benefit in regards to total loading time with compressed files. At least not for SD cards. With an actual disc drive, the benefit would be dramatic. But who uses those for real?EyeballKid wrote: ↑Mon Nov 20, 2023 4:14 am I'm assuming the idea is that you want to be able to load images directly into VERA ram for immediate display... but uncompressed image data on disk just feels... wrong.
Re: A BITMAP FILE SPECIFICATION FOR THE X16. (In progress)
I actually designed a universal binary file container a few years ago, with the intent that it could contain any media type, including bitmap files.
It might be worth a look.
The basic idea is that the file is broken into 3 block types: the header, an index, and the data
The header is just basically name and creator info, which is entirely discarded by the loader.
The index tells the loader where to start to read each block. It contains the block type, position, and length in the data file.
The data blocks are just binary data.
With this, you don't need a special data format for images; you just load the data blocks in the right locations in VERA, then set the video mode accordingly. (Ideally, by including something in the Image Metadata block to say "Use this video mode.")
Here's the spec: https://docs.google.com/document/d/1v8o ... sp=sharing
It might be worth a look.
The basic idea is that the file is broken into 3 block types: the header, an index, and the data
The header is just basically name and creator info, which is entirely discarded by the loader.
The index tells the loader where to start to read each block. It contains the block type, position, and length in the data file.
The data blocks are just binary data.
With this, you don't need a special data format for images; you just load the data blocks in the right locations in VERA, then set the video mode accordingly. (Ideally, by including something in the Image Metadata block to say "Use this video mode.")
Here's the spec: https://docs.google.com/document/d/1v8o ... sp=sharing
- ahenry3068
- Posts: 1131
- Joined: Tue Apr 04, 2023 9:57 pm
BITMAP FILE FORMAT: DRAFT REV 5.
This is still a draft. There was a lot of discussion on the DISCORD thread over the past couple days. This is REV 5 of the format.
@TomXP411. Desertfish had a similiar idea without the container header. What he suggested was almost a Video Tracker file for the X16.
That may be a great direction to go for some programs. But part of what I wanted to do with this was also make "Not full Screen" bitmaps possible. This format is meant for things like Paint Programs and also Illustrations and Bitmaps for Games or other software, (Maybe even embedded images in a Word Processing file, if anyone ever want's to write something like that.) There are a few significant changes in this one. But I'm optimistic that we can publish a specification by Week 1 in December.
ADOBE ACROBAT
LIBRE OFFICE
MICROSOFT OFFICE
@TomXP411. Desertfish had a similiar idea without the container header. What he suggested was almost a Video Tracker file for the X16.
That may be a great direction to go for some programs. But part of what I wanted to do with this was also make "Not full Screen" bitmaps possible. This format is meant for things like Paint Programs and also Illustrations and Bitmaps for Games or other software, (Maybe even embedded images in a Word Processing file, if anyone ever want's to write something like that.) There are a few significant changes in this one. But I'm optimistic that we can publish a specification by Week 1 in December.
ADOBE ACROBAT
LIBRE OFFICE
MICROSOFT OFFICE
- ahenry3068
- Posts: 1131
- Joined: Tue Apr 04, 2023 9:57 pm
BITMAP FILE SPECIFICATION DRAFT 5a
Very minor revision. Should break almost no code written to Rev 5.
Expect to call this BMX Specification version 1.0 as of 1 December 2023 unless something major comes up with it.
ADOBE ACROBAT
LIBRE OFFICE
MICROSOFT WORD
Expect to call this BMX Specification version 1.0 as of 1 December 2023 unless something major comes up with it.
ADOBE ACROBAT
LIBRE OFFICE
MICROSOFT WORD