The Palette

[Merri]

Hello, first post!

The limited colors of the old days have always been interesting to me so when I worked my way through the docs I halted on the default palette and it didn't catch my attention in a good way. Of course colors and palettes are not a fully exact science, there is always the matter of opinion and taste. I found an earlier thread on the C64 colors being off, but there are more things that I think could be improved.

What I do like about the palette is the inclusion of the whole greyscale range. And the idea to fill the remaining 224 colors with groups of 7 color gradients is a very good idea. Many of the gradients are also nice.

But then there is the critique side which I'll split to following two subjects: Commodore 64 colors, and repetition.

 

Commodore 64 colors

I don't know the origin of the current set of C64 colors, but it seems to be more of a palette fit for new pixel art than for representing what the C64 colors really were like. Most importantly it lacks the luma pairing of colors (where color pairs equal one grey when in greyscale, which also allows for "secret colors"). And there is the whole PAL vs. NTSC vs. NTSC thing (because Not The Same Color, you know) so people have used to something and you can't really ever get it right. In the PAL land we do have Pepto who has somewhat recently done further work into the colors and we now have a new Colodore palette. It is very good!

However there is of course the problem that the PAL isn't NTSC. And by looking shots from actual TVs the current CX16 colors don't really match with what NTSC produces. So, being a bit of a moron I've gone ahead and used Pepto's calculation model in an attempt to find some sort of a compromise. As if that kinda stuff would be agreed upon in the passionate C64 world ?





The top row is Colodore, the middle has current CX16, and the bottom row has a "compromise" where I've tried to adjust the colors to be more attractive and more NTSCish, but which still maintain valid VIC-II relationships between them, while also trying to get many colors as close to the current CX16 as I could. It is still quite a difference so I don't know if this trouble gains anything of real value, but it has been a fun challenge if nothing else!





Repetition



In the current default CX16 palette there are a lot of gradients that are very near to being clones of each other. There are some colors that are visually identical to the human eye such as the bright cyans. And there are also dark palette entries that are repeated multiple times over in the palette. These issues are sort of waste. I don't think this would be fitting for an 8-bit computer as the era was full of making the most of what little you have, so repeating colors should be avoided.

There is some repetition that can't be avoided: the C64 contains black, white, and three shades of grey which get repeated in the full 16 color greyscale gradient which is all the grey entries in the 4096 color palette. This means that out in the 256 color palette we can have we can have up to 251 unique colors if we leave C64 palette and the greyscales untouched. The current default palette repeats four gradients for each of red, orange/yellow, green 1, green 2, cyan, blue, purple, and magenta (I guess that is what you call those colors). This results to only 228 unique colors so we could have 23 colors more for our use. With a total of 28 repeated colors more than 10% of the palette is "wasted".

So what to do? Well, instead of just complaining about it I've seen the trouble to produce a 251 unique color palette. I can tell you it ain't easy, especially with the 12-bit limitation! I wanted to stay in the spirit of the current default palette which means creating automated set of gradients. So I wanted to be smarter about the generated gradients and this is what I've come up with:



NEW VERSION (v10) 2021-05-04 (gradients from 24 bits to 12 bits using CIEDE2000 perceived closest color)



ORIGINAL VERSION (v9) 2021-04-26



In the above palette the entire scope of hues are being used so there are four different hue regions used where there is only "red" in the default palette. What I also like about the gradients here is how they mostly mimic a lot of game palette gradients that I've seen over the years. I did do a quick test on some images and the palette works mostly very well on many old paletted DOS/Amiga games.

I don't know if this results into really anything, but I would like to see the default palette improved before fully locking it. At least I think the above proves improvement is possible!

Then finally: if interested over at CodeSandbox you can find the tool I've created for working with palettes. You can fork it ? It is currently coded to auto-generate the palette above, although with 24-bit range. The tool can open various palette file formats (including paletted image formats) and output them as well, like .ACT which is really just a 768 bytes binary file when holding 256 color palette. I'm not yet fully sure if the Commander X16 binary output is correct since I haven't started actually playing with the emulator more than trying out some of the existing software.

[rje]

Welcome, Merri!   

First, that's a neat analysis, and the reasoning looks sound.  

Now.  The authority is the FAQ.  It'll tell you what's been finalized and what's left to finalize.

But your palette is at least a good option if programs want to replace the default with a rich alternative.

Frank van den Hoef might also be someone to reach out to, since he created and manages VERA.

[desertfish]

I like your palette and gradients. While it would be nice to have such a “better” palette by default (especially for basic programs) , you can always set the colors manually in your own code. (I was already setting the pepto palette for my c64 koala image viewer)

[ZeroByte]

Somehow the various emulator versions of C64 palettes never look right to me - they look _too_ muddied, and when i hooked my C64 up to a monitor, it looked really vibrant by comparison to what I was getting in VICE while developing a game.

And I agree that the stock X16 palette is hot garbage beyond the first two rows, at least for doing 4bpp graphics. There really aren't any "plug and play"  useful palettes up there. Doesn't really matter much to me though because 4bpp art pretty much requires very specialized palettes for the assets to look good anyway, and it's pretty much a given that you're going to be rolling your own anyway, so it doesn't bother me. It does seem to have an overabundance of purple/pink/blue and a dearth of yellow, but again - I pretty much disregard it as "a starting point for people who don't want to bother making their own palettes"

Besides, I think there's no way that Dave & Co are going to change the palette now, even though it would be a simple ROM change. They're trying to go gold with this thing.

[paulscottrobson]

Not sure it's possible to replicate NTSC colour electronically. Possibly spraying a layer of mud on the screen would work.

I always though the colours were shows because they were the least awful looking on a NTSC Analogue TV.

[Merri]

Thanks for all the interest!



As for the bad emulator colors there is only one way: gotta hook that emulator to a real monitor! Although I think the emulated CRTs have become a lot better, but probably still need a true HDR caliber display to get the same kind of "shine" that CRT has. It is quite a technological luxury we live in these days as the new displays are finally catching up with CRTs final strong points! ?

I do agree on having overabundance of purples. Yet I still ended up using the complete hue circle instead of going picky since I wanted to keep the idea of being technically complete even though from another perspective that is still a bit of a failure. I'm still interested to improve this so maybe I could go ahead and have more reds and yellows, and less purple and pink tones. I used the same 16 hue angles that are used to generate the Colodore palette so maybe I could abandon that limitation and instead split it into more angles and pick more of nicer and desirable hues so that not everyone have to come up with a great night time game to make great use of the purples and blues ?

As for actually changing the default palette I don't want to push on it too aggressively even if it means we end up living with the current one for the rest of eternity. I don't like putting unnecessary strain and pressure to people working on this kind of project, but I still like the idea of contributing and being a small part of all of this. And it is a good thing that the palette is changeable so it does leave the door open for the community to figure out better general palettes.

[paulscottrobson]

11 hours ago, ZeroByte said:

Somehow the various emulator versions of C64 palettes never look right to me - they look _too_ muddied, and when i hooked my C64 up to a monitor, it looked really vibrant by comparison to what I was getting in VICE while developing a game.



And I agree that the stock X16 palette is hot garbage beyond the first two rows, at least for doing 4bpp graphics. There really aren't any "plug and play"  useful palettes up there. Doesn't really matter much to me though because 4bpp art pretty much requires very specialized palettes for the assets to look good anyway, and it's pretty much a given that you're going to be rolling your own anyway, so it doesn't bother me. It does seem to have an overabundance of purple/pink/blue and a dearth of yellow, but again - I pretty much disregard it as "a starting point for people who don't want to bother making their own palettes"



 

I use a stock palette which is loaded into 240-255  - the first 8 are the digital RGB and the other 8 I just thought might be useful, Sprites are all set up with the offset %1111 to make them use it, then the sprite converter does a best distance calculation to pick one. Seems to work reasonably well. Can use alternate palettes if you have a different environment say everything is metallic grey. I'm not sure that 256 colour palettes for pixels are really that useable, there isn't that much VRAM available. Maybe for a main sprite.

[rje]

38 minutes ago, paulscottrobson said:

I'm not sure that 256 colour palettes for pixels are really that useable, there isn't that much VRAM available. Maybe for a main sprite.

I wondered that too.  My BASIC tile-map program loads up maybe 8 land sprites and maybe 8 sailing ship sprites, and memory fills up fast.

[Fenner Machine]

A custom 256 colour palette should be very useful.

Using 4bpp with palette offset for both tiles and sprites increases the amount of images that can be stored in VRAM verses 8bpp plus has the benefit of the extra colours.

It will take planning to fully utilise, but the results should be worth it.

[ZeroByte]

I'm toying with a workflow for Aseprite that will simplify this palette stuff for me.

• 
  Work with the image as an indexed color sprite.
  


• 
  Load a 256-color palette that the project is using. This can start out as the default X16 palette, but the idea is to have a global palette for the entire project.
  


• 
  Insert 16 more colors at the beginning - these are to be "scratch colors."
  


• 
  From the main portion of the palette, copy the 16 colors of the asset's intended "color offset" row onto the cratch color slots.
  


• 
  (I made a LUA script that does this automatically)
  


• 
  Draw the sprite using only the scratch colors. If you want to play with palette swaps, just copy various color offset rows over the scratch (using the Lua script makes this a single keystroke)
  


• 
  If you change any of the scratch colors, copy them back onto whichever row they came from.
  


• 
  When exporting the asset, shrink the palette down to just the scratch colors before exporting.
  


• 
  Export the scratch palette
  


• 
  Use the img2tileset.py script (I modified this to accept more aspect ratios and to enable 4bpp color support)
  

I haven't figured out where the script was that I used to export the palette itself as a binary data file / c array / BASIC data list / etc - but that's what I'd then use to extract the palette itself.