Interpreter Idea: AWK

rje · Post by **rje** » Tue Sep 28, 2021 3:06 pm

AWK is a powerful filter DSL used for data extraction and reporting. Its fundamental structure is a set of matches and corresponding actions. It boils down, almost, to a set of if-thens based on the input stream. When you match THIS expression, do THAT. If you've ever done ladder logic, it seems kind of like that.

PATTERN    {ACTION}    PARAMETER-ASSIGNMENTS   'FILENAME'

There are shorter forms, since the condition defaults to "always", action defaults to "print the line", and parameters have defaults, so this short form means "cat":

'MYFILE'

Original Intent

"a tool that would easily manipulate both numbers and strings" and search for patterns in data.

AWK-8 ?

X16 Awk would be a stripped-down AWK. I think it can still be useful and even interesting to have tools that can do file tests, streaming filters, and report building.

Reading through several introductions to AWK, here are the features I think might work in a non-shell 8-bit Commodore setting:

PATTERNS

1 as the default pattern.

Expression evaluating to non-zero.

String matching.

Range matching?

ACTIONS

"print($0)" as the default Action.

A list of semicolon-terminated expressions.

EXPRESSIONS

RS (record separator). Integer. Default is \n.

FS (field separator). Char or char*. Default is \s.

NF: number of fields in the current record.

NR: current record number. Starts at 1.

$0 as the current record.

$1..$9 as the fields in the current record.

A reasonable stash of general purpose variables.

Integer arithmetic.

Integer comparison.

Symbol table (hashtable) operations.

Various library of function calls (like print). As many as you can fit into the binary.

rje · Post by **rje** » Fri Oct 01, 2021 2:38 pm

My current hack isn't a parser: it only uses sscanf() and strcmp() to detect some basic cases. As a result, I can catenate a file with the command

'MYFILE

And

1 'MYFILE

Note there's no trailing quote.

I wanted to use the PETSCII left-arrow, but I can't do that until I use a half decent tokenizer.

BruceMcF · Post by **BruceMcF** » Fri Oct 01, 2021 3:08 pm

Of note is that the V7 sources (from the late 70s) were released by their IP owner under an open source license, so the original awk sources are available. So rather than work up from scratch, my idea would be to work down (if necessary) from V7 awk.

Tthe PDP-11 at that time relied on what would use a 64K program, 64K data system to get around it's 64K address space limit, so it wouldn't be surprising if it could be made to fit into Low RAM.

My OTHER idea is to have a general menu based system for loading text files into linked-list strings of HighRAM segments for stdin/source/stdout type situations, so Kernel open input and output channels are used for stdin/stdout and the source is read from the text file already loaded into HighRAM. That system would run and then when a compatible binary is called, it overwrites the menu system, allowing the menu system to be written in Forth or whatever other HLL ... xForth if I am in a position to bring it to a release state, otherwise VolksForth.

BruceMcF · Post by **BruceMcF** » Fri Oct 01, 2021 3:09 pm

rje · Post by **rje** » Fri Oct 01, 2021 3:23 pm

No, that's reasonable. I worry about the heap requirements, as well as the size of the codebase -- AWK is not small.

V7 AWK is here: https://minnie.tuhs.org/cgi-bin/utree.pl?file=V7/usr/src/cmd/awk

And now it's here: https://github.com/bobbyjim/AWK-7

My question is: what would you use it for, if it were running on the X16?

Catting files is useful, but is filtering them also useful? Splitting records, and displaying columns? Counting strings? Selecting records?

BruceMcF · Post by **BruceMcF** » Sat Oct 02, 2021 1:04 pm

On 10/1/2021 at 11:23 AM, rje said:

No, that's reasonable. I worry about the heap requirements, as well as the size of the codebase -- AWK is not small.

V7 AWK is here: https://minnie.tuhs.org/cgi-bin/utree.pl?file=V7/usr/src/cmd/awk

And now it's here: https://github.com/bobbyjim/AWK-7

My question is: what would you use it for, if it were running on the X16?

Catting files is useful, but is filtering them also useful? Splitting records, and displaying columns? Counting strings? Selecting records?

Oh, there's always a use for filtering on your development host. It's true that splitting records and selecting them may be handier in general in assembly language sources than sources for Forth words, but there's always data input files, and if you include them compilation logs and error logs.