CommandTracker: Mostly ideas for a music tracker sparse file format

[kliepatsch]

1 hour ago, JimmyDansbo said:

A more general question... How did everyone else learn this stuff, have you all read music theory or have you just been playing around with synth's and trackers until you figured it out?

For me it's been a very long journey. My parents made me learn an instrument very early on, and I even had a music theory class when I was young ...

And all the synth stuff ... I don't exactly remember how it began but I made a few tutorials that got me started. I also spent a lot of time over at the One Synth Challenge (https://sites.google.com/site/kvrosc/) that gave me a lot of practice with synths.

I think the learning curve for actually *making* music is quite high. Even though it might sound very boring ... I think that experiencing what it feels like to perform music written by others is an important part in being a good musician. That way it is possible to gain an "intuition" for music. If one didn't learn an actual instrument, I think copying ("covering") an existing piece of music that one likes is a fairly good exercise and challenge. Both to get to know how a tacker works and how music works. Recreating a song is a well defined objective, but very challenging -- and that's the whole point. It will teach a lot.

[m00dawg]

I'm realizing that the original post title is no longer relevant as I'm NOT using a sparse file format at the moment haha. But anyway, here's a first quick showing of Command Tracker actually playing a simple song! It's late so I didn't have time to add in row highlights but that's next on my todo because it would be really helpful while I'm having to manually transcribe notes in hex ? I think at some point I'll need to add some sort of text file -> hex parse so I can at least create patterns in a text editor more easily.

The next bigish hurdle in playback is multiple channels as I'll have to refactor some code a bit. Right now my "playpattern" routine is really just playing the first channel (since that's all that exists in the pattern format). Once that's done I think the main components of playback will be done in terms of the basic flow. I'll need to refactor how effects work a bit (they're currently evaluated last, but I think they should be evaluated first, e.g.).

Then maybe adding in a very basic UI to be able to tab over to see more of the channels (note the arrow cue on after RS05).

 

 

[calcmandan]

And the ultimate goal for me is having a tracker on the X16 so I can make music with it. Once there is a MIDI interface, perhaps in tandem with a modern DAW; but I'm looking forward to the offline isolation of a dedicated tracker running right off the X16. I suppose there's a small part of Nostalgia at work there but not much as I do find using my modern DAW can be pretty distracting. Even if I just unplug the damn network cable, Windows still wants to make sure that I know that my online account isn't sync'd just in case I changed my mind, in addition to myriad of other nonsense. So yeah, anything that gets the closer to the finish line I'm all about!

Well with four expansion slots, I'm sure it wouldn't be too difficult for a designer to make a midi+daw card to slap in there.  

[m00dawg]

Indeed, I already have some MIDI sync designs for the GameBoy and NES. I think just a uC that runs at bus speed should be sufficient. Or something even simpler like pulses send via the controllers or user port for just doing sync. That's how the NES solution works. That will be among the last things I'll end up implementing in the tracker I think so there's plenty of time for there to be solutions to come up I think. But if not, yep it's something I'll likely tackle at some point if no one else has gotten to it by the time I'm ready to add MIDI sync into the tracker.

[calcmandan]

Indeed, I already have some MIDI sync designs for the GameBoy and NES. I think just a uC that runs at bus speed should be sufficient. Or something even simpler like pulses send via the controllers or user port for just doing sync. That's how the NES solution works. That will be among the last things I'll end up implementing in the tracker I think so there's plenty of time for there to be solutions to come up I think. But if not, yep it's something I'll likely tackle at some point if no one else has gotten to it by the time I'm ready to add MIDI sync into the tracker.

ii think it's exciting to see your project moring forward. i'm not a composer but i love music from the demoscene. i often listen to quantum radio as it plays nothing but.

Sent from my SM-T720 using Tapatalk

[m00dawg]

Awesome! Yeah long way to go yet but having a nearly usable pattern editor is a huge step forward I think (at least for me). I haven't been able to work on it this past week due to the crazy winter weather down here in Texas - rolling blackouts for the past several days (with some folks totally out of power). But I expect I'll start working on this weekend again.

Before the Texas Winterpacalypse I did manage to get a brute force save routine working. But it was "dumb" in that it just grabbed the entire order list and all patterns (whether or not they've been used) and, as a result, took something like a minute to save ? So that's not ideal! I ways trying to avoid having to write a sorting and dedup algo in assembly (to only save patterns defined in the order list) but doing that will certainly make saving faster I think, at least for a majority of songs. The real fix is a sparse format, where I'm not just grabbing an entire page of memory for a pattern, but that's probably a post 1.0 change I think.

[ZeroByte]

I've been following this thread with some interest - it's awesome to see a useful tool like this making good progress!

I'd like to throw some information into the ring. A while back, I found a page that has Rob Hubbard's Monty On The Run player completely disassembled, commented, and explained in great detail.

I think it may be useful to see how one of the masters did it back in the day. The biggest takeaway I have from this code is that patterns are completely independent of the voices - i.e. the same pattern could be used to play its notes on voice 1 for a while and then voice 2 later if you so chose. I think the idea of being able to define a pattern once and then just use it over and over is quite helpful whenever I'm noodling around in a tracker because I can just rattle off a bass line and then it's done. When each new row of the order list implies a new pattern on all voices, that means I'd have to copy and paste the same bass line into every row that uses it, and if I decided to tweak it a little, I'd have to go re copy/paste the changes into every row.

I do agree that Deflemask somehow makes the pattern selection feel very confusing where somehow Famitracker's doesn't feel that way. I can't exactly put my finger on it, but something about Famitracker doesn't have me accidentally overwriting the wrong patterns or inserting the wrong "new blank pattern" whenever I add a row to the timeline.

I was thinking that if I were to write a tracker for X16, I'd probably do something more like Hubbard's approach where each pattern is edited as a stand-alone entity, and then you just select them in the order list. The big difference here would be that each voice doesn't have its own set of patterns (pattern 0, pattern 1, pattern 2, etc) but that once pattern 1 is created, you can use it on any of the voices. For the X16, you'd obviously need to have a set of FM patterns that are only selectable for FM voices, PSG patterns for the PSG voices, etc.

[m00dawg]

Yep! I read up on Hubbard's work and it was the basis for my first attempt at a sparse file format. I then looked at how GoatTracker handled it's format and found I wasn't far off which was a good sign. My sparseformat is a lot more like Goat's than Hubbard's but the two are similar. You can find my thoughts on a sparse format here. It's not up to date to the current tracker but concepts would be similar.

To get to a 1.0 release, I'm intentionally making things simple by design. That means I had to remove features I wanted (like multiple effects per channel/row) and that many come to expect in modern trackers. So don't expect anything super fancy at 1.0. I do hope to add things like Macros as a means to work around the multiple effects things (like Adlib tracker and LSDJ) as an early post 1.0, as well as having a packed file and playback format.

The fancier features will probably be a 2.0 thing. Starting small lets me evaluate what sort of room I'll have to work with given CPU and RAM limitations. This is especially true on CPU given there's a lot of unknowns yet about how much we can do with 16 channels of the PSG before we start to eat up too much CPU to be practical.

All that, plus this being my first big assembly project, means I'm trying not to bite off more than I can chew. However, yep, I would like to see a lot of nice features in an X16 tracker and I expect to look at some of those when looking at a 2.0 release. Of note too, the project is open source so it can be contributed to, forked, hacked, etc. and I'd certainly be happy for folks to lend some hands that would be interested in doing so!

[ZeroByte]

Keep up the good work - it's definitely good to shrink the scope so you can cope. ?

 

I'm gearing up to write a nice VGM player that supports chips that can be translated and supported with the existing ones - e.g. NES APU using the PSG. I've got a long row to hoe on that front, as I've never dealt much with the RAM banking and zlib() support, disk I/O, etc. But I've kinda/sorta written a VGM player from scratch on Raspberry Pi to drive a real YM using the GPIO pins - it shouldn't be too hard to insert more chip support once I get the basics of decompressing the VGM into RAM banks, etc accomplished.

[m00dawg]

That sounds super cool! Looking forward to seeing that materialize!!