Dream Tracker

[DragWx]

That is quite lovely! Still textmode yeah? Drawn with the 1/4 squares? I'll have to see how I can do that in petdraw but yeah that's lovely!

It's all PETSCII line and corner characters, plus the filled circle character for the music note:

Code: Select all

B2 C9 B2 C9 D5 C9 D5 C9 B2 C9 C9 20 BA D2 B2 C9 B2 C9 D5 AE AE D5 B0 AE B2 C9 AA C9
C2 C2 D5 CB AB CB D5 B3 C2 C2 C2 20 D9 20 D5 CB AB B3 C2 20 AB B4 AB 20 D5 CB AA 20
B1 CB BD CD CA BD CA B1 CB 20 CA 20 20 BD BD CD BD AD CA CB BD CA AD CB BD CD D1 20
20 20 20 20 B7 C4 C4 20 20 20 20 20 20 20 20 20 B7 C4 C4

I've also attached the (tokenized) BASIC program for printing it to the screen.

[m00dawg]

Thanks DragWx! It looks awesome!

I haven't added it in - yet, but I will! I spent the weekend making the tracker functional again after breaking stuff. Cut/paste and navigation now work well, and it plays songs again Plus now it has a voice activity display. A few minor issues with pattern volume fixed.

I whipped up a demo of some of the external facing features:

https://www.youtube.com/watch?v=W_JsbCNS5Zo

[m00dawg]

This is sort of a thinking aloud dev diary update. I've been thinking about effects and I might have a solution looking for a problem or might be overthinking it. Furnace has 70-some odd effects for the X16. Not all of them necessarily make sense since Furnace caters to many many sound chips. One thing I haven't liked when using my tracker is using hex (more specific 2 characters) for each effect, so e.g.:

C-4 01 3F 1234

The 1234 is the focus. That would mean modify the 12th effect with a modifier of 34. I'm more used to letters so:

C-4 01 3F V34

Which might be Vibrato by depth 3 and speed 4. The big thing is it's one key and uses less space. Turns out a whole channel worth of space based on a mockup I made in Petdraw X 2.0:

Screenshot from 2023-07-11 20-56-42.png (25.27 KiB) Viewed 25131 times

I have to remove a border which is slightly annoying to me but fitting in an entire extra channel is kind of a big deal. Trouble is, if I use alphanumeric characters, instead of up to 255 effects, I can only get 36 (A-Z, 0-9) though can squeeze a few more in if I use some not off-putting characters (!,@,#,*). That gets me 40. How do I get more?

I had two thoughts on this. One is to use colors. So 'V' might have multiple meanings. A blue V might mean "set PWM", a light blue might mean Vibrato, and I could otherwise keep adding colors as long as I had a decent way to input that (e.g. SHIFT-V vs V or using the numbers for effect banks of sorts which map to colors). Another thought is switching to mixed case so I can use lower and upper characters, optionally in addition to color.

While the letters don't always mean something sensible I do like having meaning that's more than a 2 digit hex number. Internally the pattern data is the same so it's really a front end question, but one which could influence how many effects there are to choose from at least without having to have some special method of entry.

I say this is a problem looking for a solution because right now the tracker has no effects yet I plan on adding some soon, at least easy ones (like global speed). What affects there are any how many also depends on the sound engine and some might be more used than others, especially with macros and/or tables being a thing I want to do.

Tables would be much like they are in LSDJ. Fast effects (I would think 1 tick per vblank) where multiple effects could be set and manipulated (how many depends on memory and how much CPU I want to dedicate to the cause). Macros would be single statement multiple effects where I might expose less common effect options perhaps. Those are to get around the 1 effect per channel pattern limitation. Both are not near term things but still impact how I want to set all these things up.

Anyways that's what I've been somewhat obsessing over and thought I'd get the thoughts down, mostly for my benefit, but for folks interested, these are some of the thoughts running through my head on how I want to build out one of the core tracker features.

[DragWx]

You can also make use of the reversed characters too as an alternative to using lowercase characters. That would give 72 alphanumerics to represent effects, and you could do something simple like shift+letter or shift+number to input them into the effects column.

I think color would be good as a way to highlight what category the effect is from (e.g., pitch effect, volume effect, timbre effect, panning effect, global effect (speed change, jumps, etc)), but not so good as the main way to differentiate between multiple commands that use the same character.

[m00dawg]

I hadn't thought about reversed chars. I actually wrote them off and was planning on reusing their spots for more box art I can still do that since I can reverse the colors in VERA when drawing the pattern so yeah that's a really good point!

I kinda went down a rabbit hole trying to think about all this, particularly since it's a bit early to ponder these things. Concerto has some channel level modulation I can tap into (and he's adding more) but I also pondered what a more conventional tracker sound engine might look like when pondering all these effects and started working on structs for instrument, envelope, and channel info. All from just pondering how to organize effects That all sort of interrelates since were I do consider a tracker-engine rather than Concerto, I have to ponder how to organize the channels, particularly VERA. I had thought about having instruments be bi-timbral so each tracker channel is hard mapped to 2 VERA voices. So things like MegaMan style detune/chorus are easy and combining of waveforms can be done. This keeps the tracker at 8+8+1 as it is now but with dedicated PSG and FM channels.

I might still ponder that but I think, especially for version 1, were I to make a sound engine, making a 1 to 1 mapping makes more sense. Means more pattern-level work for making composite sounds but being particular about that kinda thing is part of the draw of using a tracker. It does make the UI more involved and means there's less room to add more sound chips down the road (e.g. a GUS style sampler card, chiptune-wavetable card, etc. - several sound expansion cards have been talked about) without crossing bank boundaries since I use 1 page per pattern making that a hard limit. At the current layout and keeping the max rows at 64 (less than ideal for 6/8 songs where 96 rows is more preferable), 25 channels is the limit. In order to get more I'd have to change how that worked (or considered using CRAM for pattern storage).

Anyways! I hadn't thought about grouping by color (e.g. pitch effects are red, volume effects are orange, etc.). That opens things up in interesting ideas! Uxx/Dxx could be reused depending on the selected effect group. So a red Dxx is a pitch down by xx, but an orange Dxx is volume down by XX. Hmm I'll have to think on that. Using 2 columns I could do this with a number modifier too. So 1xyy would be pitch effects, therefore, 1Dxx is pitch shift down. Or I guess I could use letters too (PDxx). Multiple letters I fear might get hard to read, especially as the pattern goes whizzing by which is where the colors would be interesting and I did like the idea of gaining that 7th channel (especially if having up to 25 channels).

Colors are rudely ignoring the folks who might be colorblind though so that's a big UI caveat I would want to be careful of. Ways to work around that by careful color selection though.

[m00dawg]

Pretty substantial update on this one.

I've taken some considerable steps backward as I have decided to make my own envelope-based sound engine for the VERA/PSG bits. This moves the Dream Tracker away from Concerto. Although I do want to be clear - I find the Concerto engine to be brilliant!!! It reminds me a lot of the richness of my MidiBox MB6582 synth and I'm actually quite excited to see how it will mature and plan on using it for other projects, probably in tandem with a MIDI expansion card and turning the X16 into synthesizer not unlike the MB6582. There the complexity and richness of Concerto is a really good fit for something like that and I'm actually quite looking forward to being able to use the X16 in this way and route it through my Eurorack among other things.

The problem I was running into is one of paradigms. Treating the X16 like a synthesizer is different than treating it like a soundchip. It takes extra work to make complex sounds in a C64 tracker that are very easy on the MB6582 but the flipside other things take much more work on the MB6582, such as trying to exploit all voices for different things at the same time and the ease of modulating things independently. Concerto isn't quite in that extreme but it does feel more like a synthesizer than a tracker and was causing limitations in how I wanted to do modulations common to chiptune trackers.

Given the paradigm of both how I approach trackers and how I expect other tracker composers do, it made sense to jump down to a low level and make the tracker interface more directly mapped to VERA. In many ways this aligns Dream Tracker more to Famitracker, Furnace, etc. ; although it still approaches the UI from an Impulse Tracker approach.

Currently I'm working on just VERA voices for now since FM will be different and require less fanciness I suspect.

The PSG sound engine now operates around the concept of instruments, channel registers and envelopes.

Instruments setup what end up being defaults for a channel. So when there's an instrument change in the pattern data, the engine loads up the instrument data. This includes defaults like volume, wave, pan, etc. but also includes envelope assignment (at present that is just volume but it will be wave/PW and pitch as well).

Envelopes are global and there are currently 32 and they can be up to 256 bytes in length. That was strictly to make the math easy to find an envelope in BRAM since I can use the top byte of the address to quickly lookup and env (A0 = env 0, A1 = env 1, etc.). The bottom byte is then the position.

By global I mean multiple instruments could share the same envelope and for different things. For example, instrument 00 might use envelope 04 for its volume. Instrument 01 might use envelope 04 for it's PWM. Currently envelopes are evaluated at vblank speed but I plan on adding speed flags (much like how speed works for the pattern) to slow them down if required. Each channel keeps a position of the envelope it is on so at vblank, enabled envelopes for each channel are advanced by 1 and each channel is then updated.

Loops will be support in the future as well but not at present, meaning the envelope is evaluated from 0 to 255 over and over. Envelope positions reset on a new note.

To get an idea, here are mockups of the various views I plan on implementing for instrument and env editors:

inst.png (42.54 KiB) Viewed 24611 times

full.png (37.43 KiB) Viewed 24611 times

split.png (37.11 KiB) Viewed 24611 times

What's missing are potential automation options external to vera, though it remains to be seen how much I may want to put that in an instrument vs pattern effects (including macros or tables).

At present, volume envelopes are already implemented in the engine (including enable/disable) and basic instrument support. But there's no UI to manage these at present which definitely makes things pretty limiting though folks can play around with them by adjusting values in init_song.asm. On that note load/saving of instruments and envelopes isn't present (to save them within songs or separately). Adding them to songs will be easy.

Reason being the PSG instruments and envelopes sit on their own BRAM pages. So it's just a matter of saving the pages in the right spot in the file and loading them back out. That should be fairly trivial to do (but doesn't exist yet).

[m00dawg]

I took quite a sprint of basically hammering it out everyday for most of my spare time for the past few weeks, but:

https://www.youtube.com/watch?v=GgXOJo7biUM

[m00dawg]

I ended up spending another 2 days reworking on the row-playback routine. After all that worked I wanted to have some time to, you know, USE it as a tracker. And so I did!

https://www.youtube.com/watch?v=mw8uxO5TaWM

Bonus at the end showing it working on my actual Commander X16!

At present, the UI can sometimes still be a bit quirky but everything works well enough to actually make music (and loading and saving of course). Envelopes are evaluated properly, etc. Only volume envelopes are working at present, I'll do PWM next and then likely start looking at some global effects.

I have an idea on how to do the per-channel effects, at least in part. My previous idea of switching to single alphanumeric characters is still something I want to do but am thinking about internally having a jump table for effects where it could be useful to start numerically as I go up. I could then use a lookup table to map the letters to the internal effects. Mostly just thinking aloud there. But that's one reason I wanted to start with some global effects as those would be simpler to implement (at least I think).

[Daedalus]

I have so many questions... mostly because I know very little about music, But I do know that we need music and some storage, playback standards for the X16.

Upper right of the screen, the purple diamonds... I assume they represent which of the VERA channels are currently active?
Timing: How do you time this? And are envelopes implemented by using higher resolution timing inside the note timing?
Storage: How big is that file that's playing?

[m00dawg]

All good questions! I'll answer them in groups:

* Playback and Format Standards

The de-facto music playback standard for the X16 is likely to be ZSM files. These are an extension of VGM files. My tracker doesn't currently export these but that is planned. You would use ZSM as a final export for using in a game or demo. The tracker will still have its own save file since ZSMs store data very differently. They're basically a log of writes to the sound chips. This makes playback overhead low for when wanting to play music in a game where the CPU is going to be plenty busy doing other things.

* Storage

I'm going to address storage since we're already talking about file formats. Right now the file format is fairly naive/simple:

https://gitlab.com/m00dawg/dream-tracke ... type=heads

I'm not doing any compression or sparse storage or anything like that. Most of the file format is really just storing the BRAM pages as that's where patterns live as well as the envelopes and instruments. At present each pattern is 8k. That makes the current test song a whopping 73kb. I would like to improve that but having a "full" format does make potential debugging much easier and the load/save code is very simple. So improving this is further down on the list (well below important features like common tracker effects such as pitch slide, ability to change PWM, etc.)

* Timing

So onto the diamonds and timing. Yes the upper right is for each VERA channel which is hard set to each tracker channel (so tracker channel 1 = VERA channel 1). When I add YM2151, they will likely be channels 17-24. Though trying to work out how to manage all that in the UI and I'll probably have keyboard-shortcuts to hide some channels to make navigation easier. Anyways!

The channel data is here:

https://gitlab.com/m00dawg/dream-tracke ... type=heads

The channel_flags is what I'm currently using to set active mode. The routine that makes the diamonds work just looks at the mode. Previously it looked at volume and I might still do that. The trouble is, you can have an envelope that goes to 0 volume or might want to do something like a slicer kinda sound in the pattern data by setting the volume from 00 to 3F over and over _while_ say the PWM envelope is being evaluated. That means the channel is still "active" even when it's muted. But maybe that's an overly complex approach which doesn't matter as much. The diamonds are mostly there for fun =)

Finally timing. Everything revolves around vblank at present. The song engine is evaluated every vblank (so that's the fastest speed envelopes will update). The song speed is a divisor of vblank (larger speed numbers are slower). This is mostly a throwback to original trackers but also is actually quite convenient to do since the vblank timer just counts up to song speed to know when to advance to the next row (which is probably why trackers of yore did it that way). That code is here:

https://gitlab.com/m00dawg/dream-tracke ... type=heads
https://gitlab.com/m00dawg/dream-tracke ... type=heads

MIDI is definitely plan as I have a specific use case for that (and using MIDI with Concerto as mention would be of high value when I want to treat the X16 as more of a synthesizer vs a tracker). I'm in a band (see my signature at the bottom if you're curious) and while we aren't an exclusive chiptune band but about half of what we do tends to revolve around chiptune type things somewhere. We typically combine this with external things (notably surf guitar but it could also be other chiptune solutions like the GameBoy, NES, SID, etc.). To do that reliably we need a sync. Long story short, eventually I want to support MIDI sync as well as vblank and folks can choose which one that want to use. I'm not sure how that might affect the envelopes speed. I could run those on a separate timer on a VIA if MIDI didn't give me enough clock resolution (though I believe it does or can). The row updates would just run at the BPM MIDI is sending in that case.

That requires hardware of course, which I plan on making. It might not be a full MIDI I/O board but I want something that can at least grab the MIDI clock signal and toss it onto the bus. Wanting to keep that simple and avoid CPLDs and things if I can but waiting to see how other folks solve communicating to/from the X16 bus and what microcontrollers/buffers/etc. folks end up using there. I2C is available but MIDI timing is super important so I would want to make sure it can handle that uninterrupted (at least when the user isn't doing anything else like keyboard input). If it can that would be a simple solution. Another option is to use controller input and send clock pulses that way. That's how I record the NES. Someone made a special NSF player and simple hardware to basically press 'A' to the BPM of the MIDI signal. Works really well but I would have to have internal timing for the sound engine itself since just sending the BPM wouldn't be enough resolution to evaluate envelopes.



That's probably a lot more than you cared to know But hopefully I got your questions answered up there! Since this is a bit of a build diary I thought I'd offer some fuller explanations in case you or anyone else were curious (and/or I needed to reference this down the line).