FAWK: LLMs can write a language interpreter

> I only interacted with the agent by telling it to implement a thing and write tests for it, and I only really reviewed the tests.

Did you also review the code that runs the tests?

Today, Gemini wrote a python script for me, that connects to Fibaro API (local home automation system), and renames all the rooms and devices to English automatically.

Worked on the first run. I mean, the second, because the first run was by default a dry run printing a beautiful table, and the actual run requires a CLI arg, and it also makes a backup.

It was a complete solution.

A related test i did around the beginning of the year: i came up with a simple stack-oriented language and asked an LLM to solve a simple problem (calculate the squared distance between two points, the coordinates of which are already in the stack) and had it figure out the details.

The part i found neat was that i used a local LLM (some quantized version of QwQ from around December or so i think) that had a thinking mode so i was able to follow the thought process. Since it was running locally (and it wasn't a MoE model) it was slow enough for me to follow it in realtime and i found fun watching the LLM trying to understand the language.

One other interesting part is the language description had a mistake but the LLM managed to figure things out anyway.

Here is the transcript, including a simple C interpreter for the language and a test for it at the end with the code the LLM produced:

https://app.filen.io/#/d/28cb8e0d-627a-405f-b836-489e4682822...

It's a fun post, and I love language experiments with LLMs (I'm close to hitting the weekly limit of my Claude Max subscription because I have a near-constantly running session working on my Ruby compiler; Claude can fix -- albeit with messy code sometimes -- issues that requires complex tracing of backtraces with gdb, and fix complex parser interactions almost entirely unaided as long as it has a test suite to run).

But here's the Ruby version of one of the scripts:

    BEGIN {
      result = [1, 2, 3, 4, 5]
        .filter {|x| x % 2 == 0 }
        .map {|x| x * x}
        .reduce {|acc,x| acc + x }
     puts "Result: #{result}"
    }

So here's a more typical Awk-like experience:

    ruby -pe '$_.upcase!' somefile.txt ($_ has the whole line)

    ruby -F, -ane '$F[1]' # Extracts the second field field -F sets the default character to split on, and -a adds an implicit $F = $_.split.

The money shot: https://github.com/Janiczek/fawk

Purely interpretive implementation of the kind you'd write in school, still, above and beyond anything I'd have any right to complain about.

Commendable effort, but I expected at least a demo, which would showcase working code (even if it’s hacky). It’s like someone talking about a sheet music without playing it once.

I did AoC 2021 until D10 using awk, it was fun but not easy and couldn't proceed further: https://github.com/nusretipek/Advent-of-Code-2021

A few months ago I used ChatGPT to rewrite a bison based parser to recursive descent and was pretty surprised how well it held up - though I still needed to keep prompting the AI to fix things or add elements it skipped, and in the end I probably rewrote 20% of it because I wasn't happy with its strange use of C++ features making certain parts hard to follow.

I wrote two

jslike (acorn based parser)

https://github.com/artpar/jslike

https://www.npmjs.com/package/jslike

wang-lang ( i couldn't get ASI to work like javascript in this nearley based grammar )

https://www.npmjs.com/package/wang-lang

https://artpar.github.io/wang/playground.html

https://github.com/artpar/wang

I've been trying to get LLMs to make Racket "hashlangs"† for years now, both for simple almost-lisps and for honest-to-god different languages, like C. It's definitely possible, raco has packages‡ for C, Python, J, Lua, etc.

Anyway so far I haven't been able to get any nice result from any of the obvious models, hopefully they're finally smart enough.

† https://williamjbowman.com/tmp/how-to-hashlang/

‡ https://pkgd.racket-lang.org/pkgn/search?tags=language

Yes! I'm currently using copilot + antigravity to implement a language with ergonomic syntax and semantics that lowers cleanly to machine code targeting multiple platforms, with a focus on safety, determinism, auditability and fail-fast bugs. It's more work than I thought but the LLMs are very capable.

I was dreaming of a JS to machine code, but then thought, why not just start from scratch and have what I want? It's a lot of fun.

So you are using a tool to help you write code because you dont enjoy coding in order to make a tool used for coding(a computer language). Why?