Web Embeddable Common Lisp

jackdaniel

This is just a test page. Today I've shared an information about an accepted grant proposal:

https://functional.cafe/@jackdaniel/114742776265318353

The work will go towards improving browser integration and porting to WASI.

remexre

Will this include some form of SLIME support?

jackdaniel

I didn't plan explicitly for SLIME, but loading swank shouldn't be much of a problem. The missing piece would be a bridge between a websocket and tcp I think.

jinlisp

This is another comment of the author about maxima in the browser from four months ago: I'm experimenting with WASI and the GC extension for WASM, but that's months from today if we speak about complete port (given my time capacity at the moment). Don't know if the gc extension is used in this example.

deosjr

I recently shared a project I did using Hoot, by Spritely institute. It's Guile Scheme compiling to WASM, and it works really well! See https://spritely.institute/hoot/

Latest on my project, in case you want to try it out: https://deosjr.github.io/dynamicland/whiskers.html

jinlisp

I was going to ask if maxima (a symbolic computation system) can be implemented in the browser, but it was answered four months ago [1]

[1] https://news.ycombinator.com/item?id=42853528

mixedmath

Do you use maxima? Would you say it's still worth using and learning?

jinlisp

It depends of what you need, but for example for calculus is a nice program. There is also sympy and Wolfram Mathematica. For symbolic computation I think that Mathematica is the strongest then maxima and then sympy, but sympy is based on python and I think it will get stronger. If you need numerical computation then there is octave or matlab or julia.

mrbungie

For symbolic computation there is also Maple (*shudders at the thought*).

johnisgood

Yes, I think it is worth learning. GNU units is worth learning, too. You know these math riddles? You can solve it using GNU units alone. :D

jackdaniel

Btw, eval (ed "wecl.lisp") to see some interesting function definitions, like canvas or webgl access drafts.

tempodox

> Are we consing yet?

  > #'cons
  #<compiled-function CONS 0x43cee0>

That's a definite yes :)

bevr1337

Why was LISP not one of the first languages able to compile to WASM? I've been curious ever since the awesome WASM project started tracking different projects. They use emojis for progress and LISP has been a hatching egg.

dmkolobov

If I had to guess:

1. lack of native GC, you had to roll your own by providing a runtime.

2. lack of tail-call elimination in V1 of the spec. This essentially forces you to trampoline everything.

no_wizard

It sorta has been. As a language design it is already native via the WAT[0] which compiles to web assembly.

[0]: https://webassemblyman.com/wat_webassembly_text_format.html

bevr1337

Do you know why only sorta? My understanding is that only toy implementations exist right now but nothing production ready. I'm poking around Google and GitHub and not finding any new information.

> Implementing Lisp onto WebAssembly is so obvious that you may wonder why somebody did not have this idea long ago.

I did find this on the Femto GitHub and got a chuckle. Yes, dear author, that is exactly the piece of information I want explained. Too funny.

b0a04gl

tbh imo the slowness isn't the bottleneck. browser bytecode's always gonna trail native, that's a given. what matters is you're feeding live eval into static pages. lisp as a runtime inside the web, not around it.

if wasm interop clicks, even at shared memory level, lisp can directly drive ui updates. state changes, you eval, dom shifts. pure control flow without detour

jinlisp

When executing (time (loop for i below (expt 10 8) count t)) it takes a long time, sbcl takes less than a second on this. So not useful when speed is required. More: (time (loop for i below 1000000 count t) takes 7 seconds on my computer, so counting to (exp 10 8) would take 700 seconds, and sbcl do that in 0.1 seconds, so in this example the ratio is 7000 times slower than sbcl.

jackdaniel

Bytecodes compiler used in this build from repl is one-pass with very little optimizations, so it is not surprising. Natively compiled code is much faster.

jinlisp

Should it be possible to implement an optimization compiler? so that compiling the code in the web page produces results similar to ECL or sbcl?

jackdaniel

With enough code - yes. But not right now. You may precompile to native though.

potholereseller

Apparent source code, though last update is January 1, 2025: https://fossil.turtleware.eu/wecl/timeline

HexDecOctBin

Is WASM a good target for REPL-driven programming? I haven't studied it in detail, but I do remember that it was Harvard architecture. Would that mean that code can't be updated at runtime in real implementations?

runlaszlorun

I'd use the analogy of current dynamic language VMs being written C. You're not modifying structs at runtime or creating C functions as a user of the dynamic language. But the creators of the VM are providing as thin a layer as possible on top of that to give the runtime behavior it's dynamism.

There's an often repeated line that WebAssembly isn't really "assembly" and it's not "web". There's a lot of truth to that. I dove in hoping for a cross platform, web enabled assembly. It looks a lot like an assembly but I find it's relatively few restrictions like lack of a true jump command rippling out more than one would have expected. It's also sortuva stack machine and sortuva register machine at the same time.

It does share a lot with the internal VMs in many places like the JVM, Python's VM, .net, etc.

DonHopkins

"I'm a little verklempt. Talk amongst yourselves. I'll give you a topic: WebAssembly is neither Web nor Assembly. Discuss." -Your Host, Linda Richman

https://www.youtube.com/watch?v=oiJkANps0Qw

jackdaniel

ECL features native code and bytecodes vm for targets without incremental compilation support - both native and bytecode can be freely mixed at runtime.

That means that you may interactively use repl to call functions that were compiled ahead of time and to add new functions even on uncooperative targets.

After you've finished prototyping you may compile the finished library to native ahead of time.

DonHopkins

It is, actually! I didn’t realize that until I read the handwritten WASM s-expression source code of WAForth.

Languages like Lisp or Forth that run on WASM — or even handwritten s-expression-style WASM code like in the WAForth kernel — can dynamically compile and assemble new WASM bytecode and data. They can then call back to the JavaScript host to create and link entirely new WASM modules on the fly, and immediately call into them from the current module or others. A single WASM context can juggle many dynamic modules, large and small, all generated and linked at runtime!

So you can compile new code in Forth or Lisp and immediately link and call it using indirect cross-module calls. It’s not the fastest way to call into WASM, but a metacompiler — like Mitch Bradley’s ForthMacs / OpenFirmware — could recompile the whole system into one efficient, monolithic WASM module that uses direct calls.

Here's the author's blog post about WAForth:

A Dynamic Forth Compiler for WebAssembly

https://el-tramo.be/blog/waforth/

https://news.ycombinator.com/item?id=34374057

DonHopkins on Jan 13, 2023 | parent | context | favorite | on: What the hell is Forth? (2019)

This is a great article! I love his description of Forth as "a weird backwards lisp with no parentheses".

https://blog.information-superhighway.net/what-the-hell-is-f...

Reading the source code of "WAForth" (Forth for WebAssembly) really helped me learn about how WebAssembly works deep down, from the ground up.

It demonstrates the first step of what the article says about bootstrapping a Forth system, and it has some beautiful hand written WebAssembly code implementing the primitives and even the compiler and JavaScript interop plumbing. We discussed the possibility of developing a metacompiler in the reddit discussion.

I posted this stuff about WAForth and a link to a reddit discussion with its author in the hn discussion of "Ten influential programming languages (2020)":

https://news.ycombinator.com/item?id=34056735

Yes, I agree FORTH should be probably be on the list, at least if the list was a few languages longer, for as influential as it's been.

WAForth for WebAssembly is beautiful and modern!

https://github.com/remko/waforth

It's a lovingly crafted and hand written in well commented WebAssembly code, using Racket as a WebAssembly macro pre-processor.

I learned so much about WebAssembly by reading this and the supporting JavaScript plumbing.

The amazing thing is that the FORTH compiler dynamically compiles FORTH words into WebAssembly byte codes, and creates lots of tiny little WebAssembly modules dynamically that can call each other, by calling back to JavaScript to dynamically create and link modules, which it links together in the same memory and symbol address space on the fly! A real eye opener to me that it was possible to do that kind of stuff with dynamically generated WebAssembly code! It has many exciting and useful applications in other languages than FORTH, too.

Lots more discussion and links in the reddit article.

But here's the beef, jump right in:

https://github.com/remko/waforth/blob/master/src/waforth.wat

Reddit /r/Forth discussion of WAForth:

https://www.reddit.com/r/Forth/comments/zmb4eb/waforth_wasmb...

remko:

Author here

If you can't be bothered to install VS Code, you can have a look at a standalone version of the example notebook (in a 26kB self-contained page).

And if you're planning to go to FOSDEM 2023, come say hi: I'll be giving a talk there on WebAssembly and Forth in the Declarative and Minimalistic Computing devroom.

DonHopkins:

I really love your tour-de-force design and implementation of WAForth, and I have learned a lot about WebAssembly by reading it. Never before have I seen such beautiful meticulously hand written and commented WebAssembly code.

Especially the compiler and runtime plumbing you've implemented that dynamically assembles bytecode and creates WebAssembly modules for every FORTH word definition, by calling back to JavaScript code that pulls the binary bytecode of compiled FORTH words out of memory and creates a new module with it pointing to the same function table and memory.

WebAssembly is a well designed open standard that's taking over the world in a good way, and it also runs efficiently not just in most browsers and mobile smartphones and pads, but also on the desktop, servers, cloud edge nodes, and embedded devices. And those are perfect target environments for FORTH!

What you've done with FORTH and WebAssembly is original, brilliant, audacious, and eye-opening!

I'd read the WebAssembly spec before, and used and studied the Unity3D WebAssembly runtime and compiler to integrate Unity3D with JavaScript, and I also studied the AssemblyScript subset of TypeScript targeting WebAssembly and its runtime, and also Aaron Turner's awesome wasmboy WebAssembly GameBoy emulator .

I first saw your project a few years ago and linked to it in this Hacker News discussion about Thoughts on Forth Programming because I thought it was cool, but it's come a long way in three years, and I'm glad I finally took the time to read some of your code, which was well worth the investment of time.

Until reading your code, I didn't grasp that it was possible to integrate WebAssembly with JavaScript like that, and use it to dynamically generate code the way you have!

Also, the way you used Racket as a macro assembler for WebAssembly was a practical and beautiful solution to the difficult problem of writing maintainable WebAssembly code by hand.

Even for people not planning on using FORTH, WAForth is an enlightening and useful example for learning about WebAssembly and its runtime, and a solid proof of concept that it's possible to dynamically generate and run WebAssembly code on the fly, and integrate a whole bunch of tiny little WebAssembly modules together.

Playing with and reading through your well commented code has really helped me understand WebAssembly and TypeScript and the surface between them at a much deeper level. Thank you for implementing and sharing it, and continuing to improve it too!

remco:

Wow, thanks a lot, I really appreciate that! It makes me very happy that I was able to get someone to learn something about WebAssembly by reading the source code, which is exactly what I was going for.

[More links and discussion of WAForth, WebAssembly, and Forth Metacompilers:]

https://www.reddit.com/r/Forth/comments/zmb4eb/waforth_wasmb...

Here's the author's blog post about WAForth: A Dynamic Forth Compiler for WebAssembly