New community dev tool uploaded: 8sh

[rje]

8sh

View File

---

https://github.com/bobbyjim/x16-8sh

This is the early early stage of an attempted "shell" for the X16.

By definition, a command shell is an interpreter that exposes the system.  In practice, the shell exposes the system through an immediate-mode scripting language, which can also be executed from a file.  The Commodore computers' boot mode is a kind of "BASIC shell".

This shell currently does almost nothing.  I am slowly working out a set of mid-level operations for it, and plan to add the shell's command inventory on top of that. Then, I'll add the ability to run full scripts from file.  Then, I'll add the ability to "pipe" the output of one script into another.  In this manner, I hope to build up a small inventory of useful utilities.

 

 

 

 

---

• 
  
  
  Submitter
  
  
  
  
  
  
  rje
  
  
  


• 
  
  
  Submitted
  
  
  
  
  11/11/20
  
  
  


• 
  
  
  Category
  
  
  
  
  Dev Tools
  
  
  

 

[rje]

Well I can't run this thing from the "Run This Now!" button ? and I don't know why.

 

[SlithyMatt]

1 hour ago, rje said:

Well I can't run this thing from the "Run This Now!" button ? and I don't know why.



 

Try renaming the file to have a .PRG extension.

[rje]

Thanks Matt!  That didn't work either, but I'm sure it's just something silly.

 

[rje]

So I've poured some time into 8sh, and since I'm doing it carefully, of course that means it doesn't "do" anything yet.

It has three main pieces, in various states of completion:

(1) A working tokenizer which recognizes about half of the tokens I want.

(2) A compiler that doesn't work yet.  It will orchestrate the tokens and emit bytecodes.

(3) A bytecode interpreter ("VM") that currently only knows nine opcodes (out of 40-ish).

 

At some point, the compiler will *just* work, and then the whole chain will show signs of life.

[rje]

Wow, writing C on an 8 bit machine is ... well it's tricky I think.

I started out with a smallish but typical interpreter project, in C (https://craftinginterpreters.com/).  And I hit walls as memory got corrupted -- and I hit the limitations of C on 8 bitters, such as the small max size of structures, and the tiny stack (only return small things like pointers... or better yet, use global variables as much as possible), and how apparently easy it is for even global data to get lost and/or corrupted when a program gets too clever...

*** GOAL ***

Each time I hit a wall, I had to ask myself what my END GOAL was.  And I think it's not changed:

A "shell interpreter"...

with...

• 
  dynamic variables (numbers and strings... perhaps as C-shell-style arrays),
  


• 
  a pile of operators,
  


• 
  flow control (if, foreach, and while)
  


• 
  script "chaining" / output redirection (pipes, sort of)
  


• 
  5 byte floats (not a priority)
  


• 
  maybe functions (not a priority)
  


• 
  tokenizer => compiler => bytecode interpreter
  

I use Banks for loaded scripts, tokenizations, and as a command line buffer.

*** PROGRESS ***

I'm busy paring down the tokenizer.  Each time I think I've got it down, something else happens.  So now I'm shoving input into a Bank and tokenizing from there.  I'm about ready to scrap the Token struct and instead shove token information into Banked RAM.

 

 

 

[rje]

So Bank-based scanning is a success: input is stored in Bank 1, and the tokenizer works on that.  Six bytes of metadata per token is generated (type, length, start position, and line number) and stored in Bank 2.  Bank 1 is unmolested. Thus I can theoretically juke around the tokenized list by multiplying token number by 6; when I need to fetch actual data, the token metadata points me to the substring from the source bank.  A handy little utility lets me get and put strings, ints, and bytes in and out of banks.

Getting tokenization out of structs and into banked RAM has really paid off. 

Next step is getting the compiler to read from the banked token stream.  I think decoupling the tokenization from compilation will also pay off here as well, since the compiler already does a lot of work.  I am also now thinking that perhaps the compiler ought to write bytecode to a third bank.

Banked RAM is turning out to be more than just a luxury: it's becoming a "relatively cheap heap" for my program.  It's like the RAM banks are already allocated hunks of dedicated memory.  It's an extreme luxury.  It's also giving me a lot of cheap workspace, something I wouldn't really be able to get normally on machines like the Commodore 64.

And yeah, it's gonna all be glacially slow compared to assembly language.  If I get what I want out of this thing, I won't care.

 

 

[rje]

For the moment, I will pretend that the X16 is nimble enough to manage an ever-growing bytecode list.  It may be that this memory will also get messed up; if so, then I will change this system to write to a third RAM bank.  This will also probably simplify the program more.  For now though I will use what I have already written: if it ain't broke...

 

[rje]

Wow, always watch for those off-by-one errors.

 

[rje]

PROGRESS

Arithmetic expressions are compiled and interpreted, but only the first full expression.

CURRENT WORK

I'm extending the engine to recognize more than just integers.