Commander X16

Rough day. Good news is that another Code Cleanup is complete, but the bad news is that I've reimplemented  Move, Warp, Scan, and Fire as far as I could take them - and was about to see how the Regional map printed out on generation but ran into the memory problem again.



I commended out Sector - expecting to use High Ram for that, later, but unfortunately - I was so close to finally getting a shot of the revealed map instead of ???? when I ran out of memory again - this time inside a lowly subroutine to print the bottom bar. Out of Memory again.

This cannot continue. I need a better way to store everything, or I need to find new ways to handle individual characters or not store as many strings.

Case in point - the way screens are drawn.

These are stored as strings. Strings that I really don't exactly know where on the X16 low memory they are stored on the stack or when they are garbaged. The top line is a 80 character string. The bottom three bars are 3 80 character strings. The title screen is therefore some amount of string descriptors and some amount of arrayed characters in the assigned memory space - resulting in about 4234 bytes, or otherwise 10% of our available memory space before whenever the garbage collector gets to remove the ones seen in the screen.



This is my best opportunity now to handle this without understanding how to write to a specific bank as easily as I write to wherever. I'm clearly at a stopping point, and I need to crunch my memory usage because what is currently allocated shouldn't be swapped out of memory unless I want a LOT of very interesting glitches or bugs to appear down the road or find some way to always safely process-out the swapping of banks between calls. It's time to stop coding and start reading, as the memory limit is preventing me from testing and making pushes forward.

Quote

String storage in C64 BASIC

C64 BASIC uses 3 bytes for each string, which are stored in the 5 byte area of a simple variable located in the variable area or just takes exactly 3 bytes per element in case of a string array. This structure, called string descriptor, consists of one byte containing the length of the string (hence the maximum of 255 bytes) and of two bytes forming a pointer to where the string is kept on the string heap.



With statements such as, 10 A$="A STRING" or strings that are filled by a READ statement, the string descriptor points to program area where the statement is located so no space for the string itself has to be used. Other strings such as those built with an INPUT statement, returned by string functions or those created by joining two or more strings together are stored in the area of memory between the array area and the highest memory used by BASIC which is commonly known as the "string heap". The string pointer points to the start of this string.



Whenever a string that points to string heap is changed, its pointer is changed and the former string that was formerly referenced by this pointer is unused now on and becomes "garbage". To get the new string present on the string heap a new memory allocation will be made - even if the original string would be large enough to contain the new string. In case of statements like A$=B$ where B$ already points to the string heap then rather setting A$ to point to the same string, a new copy of B$ is created (as a result of the generalized expression evaluation) on the string heap where A$ will be pointing to.

https://www.c64-wiki.com/wiki/Garbage_Collection

Oddly enough - a potential answer in Garbage collection.

So, there is a solution here which might have diminishing returns very quickly. Every time I make a string in code, the descriptor points to the program, where it's already been stored. This gets large very quickly, but what I'm fuzzy on is - does it ever get UNallocated, for example, when exiting the scope of a GOSUB after RETURN by the garbage collector?



If it is, then the cost of pre-allocating often used strings is cheaper than defining them in the code (alike above). That is one direction I could take.

Sadly, I don't know what exactly I'm running out of - There are segments of memory dedicated to pointers and segments that contain their contents. Given the state of the program right now, I have have lot of strings defined in code.

What I think I need right now is a way to get the total size of a segment of memory (the string content, most likely) and give myself an in-game meter in order to watch.



In the midst of that little task -  uh oh:





EDIT 2:  ......................................Comments. The memory usage in a program includes the space taken by remarks. Wow. Just wow. After deleting some documentation, I was back under the limit and able to play. I have to delete and compress all of the comments in this file, and I know there are still thousands more lines to this program when it's done.

Well - that is pretty devastating. I'm not even a third of the way done.



EDIT3: Well, after some much-needed sleep - the answer lays within the ability to save program code space by swapping files and eliminating code from the main file to another file to be loaded and executed before popping back to the main loop. I know how to generally do this elsewhere, but not in BASIC V2 on the X16. How this is done without ruining the process or the program counter and making it as easy as functional programming - well - it feels a bit out of my league right now, but I'm not going to wimp out. I just need to understand how it's done cleanly within the loop of DRAW, INPUT, UPDATE, and returning to the main screen so the user can play.

I could use an example game that is using multiple files to manage its game state like this and how I could package it on a file system.

At this point, I've hit the memory ceiling and cannot code further unless I solved this problem.

The program is too big. The best thing to do is to LOAD and RUN a separate program and RESUME where the program left off when the subprogram finishes executing. This is not something I know how to do in BASIC V2, though it was easier to solve in SMILEBasic, where I had 6 file slots and would run the main loop in 1, data and core in 2, Map and Input in 3, and Content loading and swapping in 4 and 5. If I understood the code necessary to do this in this language, I could do this. I'm not a stranger to The Game State Machine.

But, it seems my general design in this one-file-wonder that takes place mostly on a single screen is going to have a lot of trouble, as the game never really changes state. How do I save the most code space loaded into memory for execution and smoothly drop into and out of subprograms? I definitely could use some wisdom and instruction on this.



Currently, the one-file-wonder is organized as so:

Quote

LINE NUMBERS:



    - 0 series    - Boilerplate, Common, and Core Program.

    - 1000 series     - Common, and Core Program.

    - 2000 series     - For Main Loop and Graphical/Text Functions/Subs

    - 3000 series     - For all GUI and Screen Functions/Subs

    - 4000 series     - Ship Systems Implementation

    - 5000 series     - RESERVED - COMBAT AND TURN MECHANICS

    - 6000 series     -

    - 7000 series     - Map Handling Subroutines.

    - 8000 series     -

    - 9000 series     - IN FLUX

    - 10000 series     - Command Implementations

    - 20000 series     - RESERVED FOR STRING AND TEXT CONTENT

    - 30000 series  - Termination and Error Handling.

    - 40000 series  - Data Tables

The current tasking for this involves removing the 3000 series screens, since they are entirely separate screens and terminating game screens. This would similarly be done with anything currently occupying blocks that are just static data initialization. Sadly, I don't know if I could get away with it, since it DOES mean all that program still needs to be loaded into the precious Low Memory.

A more elegant design might have me able to take the longest Subroutines and create them as programs - effectively turning a loader into the equivalent to a GOSUB. That would be best. This means I have unlimited power on the filesystem, since our filesystem has no conceivable limit in size other than the media.



I am not experienced enough to really understand good, clean, organized design on this platform, so at this point I could use some wisdom and examples. In the meantime, I pitter away at small tasks and spend time cleaning code in preparation. I want a 0-1999 Segment that I can copy paste into anything, and I'm fairly close to having that complete as soon as a few more code cleanups are finished.

I had this problem with my Traveller Trader game -- the star chart just takes up too much space.

The solution is to shove it into a high RAM bank.  I'm slowly working on that.

 

You will probably need to move all your data (text and otherwise) to banked RAM. You can put it in a separate file that is loaded straight into banked RAM, which can be done at the beginning of your program. That way, your BASIC program is mostly code.

On 9/14/2020 at 8:43 PM, SlithyMatt said:

You will probably need to move all your data (text and otherwise) to banked RAM. You can put it in a separate file that is loaded straight into banked RAM, which can be done at the beginning of your program. That way, your BASIC program is mostly code.

Matt's right.  Your main program can start with something like 

10 poke $9f61,1 :rem point to RAM bank 1
20 load "screendata.bin",8,1,$a000 :rem load screen data
30 poke $9f61,8 :rem point to RAM bank 8
40 load "mapdata.bin",8,1,$a000 :rem load the map
	

Apparently, if the file is larger than 8K, then the loader will simply move to the next bank.  So you could load your entire map into banked RAM with one load.

 

When I was storing ship definition data in BASIC, I would store them in strings that looked like data structures (but of course weren't):

rem -------------------------------------------
rem QSP + strength + cargo + alignment + name
rem -------------------------------------------
dim qs$(17)
qs$(0)=  "S-AL222  3IMURPHY"
qs$(1)=  "S-AA223  5ISERPENT"
qs$(2)=  "J-AL222 20 ASAR"
qs$(3)=  "A-BS111 80 BEOWULF"
qs$(4)=  "A-BS121 60 MARAVA"
qs$(5)=  "K-BA122  6 TARKINE"
qs$(6)=  "Y-BS121 20 BAKAAL"
qs$(7)=  "P-DA643 40*AZ ALRRAK"
qs$(8)=  "R-DA111200 MARCH"
qs$(9)=  "G-DL713 10AKHOSAA"
qs$(10)= "E-DU523  2IGAZELLE"
qs$(11)= "M-FU131120 BRILLIANCE"
qs$(12)= "E-FS413 24ZSESAVETL"
qs$(13)= "G-FU423  3ZSHIVVA"
qs$(14)= "C-HU333 85 BROADSWORD"
qs$(15)= "G-HS443 30VSE KOEZ"
qs$(16)= "F-KS131400 SUSA"
qs$(17)= "R-TB432250 LEVIATHAN"

So for example, when a Type 17 ship was encountered, I'd find its definition in this table, and extract the fields using left$(), right$(), mid$(), and val().  It's name ("Leviathan") is stored at mid$(qs$(17),12), for example.

It's really rough on the heap, I suppose.

 

52 minutes ago, rje said:

Matt's right.  Your main program can start with something like 

10 poke $9f61,1 :rem point to RAM bank 1
20 load "screendata.bin",8,1,$a000 :rem load screen data
30 poke $9f61,8 :rem point to RAM bank 8
40 load "mapdata.bin",8,1,$a000 :rem load the map
	

Apparently, if the file is larger than 8K, then the loader will simply move to the next bank.  So you could load your entire map into banked RAM with one load.



 

Actually, it's even easier. You can just use the third argument to LOAD to specify the starting bank:

Quote

10 LOAD "SCREENDATA.BIN",8,1,$A000



20 LOAD "MAPDATA.BIN",8,8,$A000

 

Okay - so this is basically LOAD "FILE", DEVICE, BANK, ????

And - to close that circle - how do you access things that you put in that bank? (I have a feeling my question thread now contains this - just humor me)

If I can get this understood and successfully demoed for myself, I can redesign the program and REALLY have some room to work with, as well as organize the code much better with full documentation. I'm not used to being so formally punished for doing (what I consider to be) a good job! XD

25 minutes ago, Starsickle said:

Okay - so this is basically LOAD "FILE", DEVICE, BANK, ????

Yep, that's it.

25 minutes ago, Starsickle said:

And - to close that circle - how do you access things that you put in that bank? (I have a feeling my question thread now contains this - just humor me)

Ah, that's the harder part in BASIC. Unfortunately, BASIC doesn't support pointers, much less far pointers, so you'll still need to poke the bank number and peek at addresses to read the values back. That means one byte at a time, so even if you are storing strings, you'll need to deal with them as arrays of character indices.