Skip to content(if available)orjump to list(if available)

Clojuring the web application stack: Meditation One

elchief

Metabase is written in clojure, if you want to see the source code of a large web app

https://github.com/metabase/metabase

adityaathalye

+1

NASA's Common Metadata Repository is worth exploring too https://github.com/nasa/Common-Metadata-Repository

It is a neat example of how an org can structure and manage multiple projects and services in a single git repository. They've use Leiningen to achieve their objective.

> The Common Metadata Repository (CMR) is an earth science metadata repository for NASA EOSDIS data. The CMR Search API provides access to this metadata.

> Building and Running the CMR

> The CMR is a system consisting of many services. The services can run individually or in a single process. Running in a single process makes local development easier because it avoids having to start many different processes. The sections below contain instructions for running the CMR as a single process or as many processes.

(edit: add relevant context for quick reference)

geokon

wow I had no idea Nasa used Clojure. I do remmeber them using quite a bit of Java so it's not terribly surprising

ramirond

Thanks for the shout! I recommend this video about our Clojure journey: https://www.youtube.com/watch?v=vUe3slLHk20

We are also hiring Clojure devs: https://www.metabase.com/jobs

adityaathalye

The ongoing discussion for Biff [1] prompted me to re-share my post because I'd like more people to understand this "other way". Outside Clojureville, it is not obvious most of these Clojure "frameworks" are not monoliths.

The consummate Clojurist's default (and very normal-feeling way) to build a web application (or any application for that matter) is to roll their own web stack from production-grade libraries.

Of course, this state of affairs is a double-edged sword, just like is true for traditional web frameworks. In my post, I try to go into the whys and the wherefores, building upward from first principles.

[1] Biff – a batteries-included web framework for Clojure https://news.ycombinator.com/item?id=44037426

andersmurphy

A fantastic post! Enjoyed re-reading it.

I'd say the nice thing with the Clojure way of building your own stack is it becomes quite easy to swap parts out. On a previous project we swapped out our web server three different times with minimal changes (jetty -> aleph -> httpkit) as for the most part they all shared the same interface.

After a while you get good at seeing where you want things to be configurable and where you don't. It also gives you the confidence to roll your own micro stack/framework which means you are not dependent on third party aggregates to adopt new features.

adityaathalye

Thank you for the kind appreciation. Made my day :)

Yes; next to the sheer stability of parts, their fungibility is a business-critical feature of the Clojure ecosystem. Of course said fungibility does not magically manifest. However the effort to get there is "not much", I'd say. The use of "system" libraries, with some well-reasoned module design brings it pretty close to magic.

As in the post, a fungible, production-grade part can be just a multimethod (e.g. the router in the post). Why? Because "production" comes in all sizes. A small SaaS with a few hundred customers may chug along happily with a bunch of functions.

yakshaving_jgt

This suggests to me that “production grade” isn’t much of a qualifier at all then.

You could just as well say that PHP has “production grade” functions.

gehrman

Your comment reminded me of this talk https://www.youtube.com/watch?v=MZy-SNswH2E.

The part about building just the functionality you need, using the bare minimum libraries etc stuck out.

yakshaving_jgt

> On a previous project we swapped out our web server three different times

Why is this desirable?

On all my projects over the past 10 years, I’ve swapped out the web server exactly zero times because the one I have works just fine. The parent comment describes these components as “production grade”, but then if that’s the case, what could be the reason for swapping them out other than self-indulgence?

andersmurphy

It's not. But, the fact that we could and it was straight forward still amazes me.

In our case the first time we needed to, we needed SSE and at the time there was no ring-jetty async interface (it's a long time ago so I'm forgetting the details) so we moved to aleph. Much later, we wanted to try out http-kit (self-indulgence) as we were operating behind a proxy anyway for performance reasons and it made a significant difference.

If we'd just started with http-kit that would have been fine. I guess it comes down to what features you need.

Also I 100% agree it's something ideally you would want to avoid a in the case of databases for example so much performance is left on the table because for some reason we want to be able to swap between SQLite, postgres and mySQL. Which in practice you never want to do.

adityaathalye

> Why is this desirable?

Generally, yes one would not want to swap out their web server willy-nilly...

Yet, this is one of those "YAGNI in 99% of your use cases", but when that 1% use case arises, a server swap would be far more desirable than a whole framework shift.

So while self-indulgence can certainly be a motive (and why not? as long as everyone's having a good time), may I offer a few more charitable reasons for this:

- programming API ergonomics

- performance

- application runtime model (servlets -> embedded server)

- security model

- application server features (websockets, comet?)

- binary size

- server configuration niceties

etc...

That said, a developer only has flexibility if it is built in from the get go.

A counterfactual would be to consider the set of developers who have had to put in ugly hacks because they can't just rip the web server out of the framework of choice they are locked into.

(edit: bullet list formatting)

wink

The problem with this approach is that (esp. for hobby projects) updating stuff is a bit tedious. Let's say you have a relatively bare bones project in Rails or a PHP framework you have a couple of dependencies that people usually use together, so upgrading can be quick and painless.

I've now had it several times in the years-long lifespan of small clojure web projects that people have moved on and the thing (framework-ish) basically doesn't exist anymore and going by the issues it only had like 10 users in the first place.

It's not the end of the world, and fortunately there's not a lot of needless churn, but I guess I would prefer to have this "I am trusting project x and I only have to care about their releases (pre-testing all the moving parts) and then my 5 dependencies" and instead I have 20 dependencies/moving parts for my web app.

Yes, I'm lazy and I don't think it's a problem in an env where you have a proper dev workflow anyway.

oldpersonintx2

Hot take

Clojure seems to use 10x the memory and run at 1/5 the speed of something I would write in Go

diggan

Hot take: People on the internet make wild claims without any sources or data to back up what they're saying, and they think that's OK because they preface it with "hot take:" so whatever you reply, doesn't matter, because it was supposed to be a "hot take" that the author cannot really defend when needed.

ertucetin

Nope, please do a benchmark. It's almost saying Go is 5 times faster than Java.

librasteve

a very lyrical post, i will reread at my leisure and try to apply the lessons to https://harcstack.org

that’s HTMX, Air, Red & Cro btw

that said … I am a true believer in HTMX for the right amount of UX dynamism and I don’t initially get solves that piece

andersmurphy

If you need the next level of UX dynamism and or realtime updates and or multiplayer. I've handled a billion checkboxes[1] with clojure, sqlite and datastar (realtime hypermedia) just fine.

[1] https://checkboxes.andersmurphy.com

adityaathalye

Thank you for the kind appreciation.

Following writing advice or post structuring guidelines is not in my job description at evalapply.org Luckily, Michael Hamburger offered a legitimate excuse in his classic (so I'm told) essay, "An Essay About Essays": https://substack.com/@bombaylitmag/p-162583447

null

[deleted]