Writing toy software is a joy

The few times I've used LLMs as question answering engines for anything moderately technical, they've given subtly-but-in-important-ways incorrect information such that taking them at face value would've likely lost me hours or days of pursuing something unworkable, even when I ask for references. Whether or not the "references" actually contain the information I'm asking for or merely something tangentially related has been rather hit or miss too.

The one thing they've consistently nailed has been tip-of-my-tongue style "reverse search" where I can describe a concept in sufficient detail that they can tell me the search term to look it up with.

> Now with LLMs, I simply prompt the same with a little bit more of context "pros and cons of using mysql vs mongodb when storing photos. Link references".

In near future, companies will probably be able to pay lots of money to have their products come up better in the comparison. LLMs are smart enough to make the result seem "organic" -- all verifiable information will be true and supported by references, it will only be about proper framing and emphasis, etc.

I use it the same way. The feeling is that I'm back in ~2010 when Googling stuff felt like a superpower. I could find anything back then.

Of course, it didn't last long, and trying to Google now is an exercise in pain and frustration. Lots of people have complained about the various things Google and marketers have done to get there, idk, I just don't like how it works now.

Top LLMs feel amazingly good at rapidly surfacing info online, and as I go through the references they're usually pretty good. I guess the same forces as before will apply, and there will be some window of opportunity before it all goes away again.

> Am I the only one using LLMs as if they were a search engine?

Nope, you're not alone as I also do this. I'm also not using any AI IDE's (yet).

It's funny, I actually recently failed a live technical interview where I was using my LLM of choice to answer Google-like queries instead of an IDE like cursor. The interviewer told me that he had actually never seen anyone use AI for coding like that before. Up to that point, I assumed that most coders were mainly using AI as a search engine and not necessarily using AI IDE's yet. Are we really that rare?

I do this. But the killer usecase for me is writing all boilerplate and implementing some half-working stuff keeps my attention on the issue which makes me able to complete more complex things.

A recent example is when I implemented a (Kubernetes) CSI driver that makes /nix available in a container so you can run an empty image and skip a lot of infra to manage.

I talked to it a bit and eventually it wrote a Nix derivation that runs the CSI codegen for Python and packages it so I could import it. Then I asked it to implement the gRPC interface it had generated and managed to get a "Hello World' when mounting this volume (just an empty dir). I also asked it to generate the YAML for the StorageClass, CSIDriver, Deployment and DaemonSet.

So LLM left me with a CSI driver that does nothing in Python (rather than Go which is what everything Kubernetes is implemented in) that I could then rewrite to run a Nix build and copy storepaths into a folder that's mounted into the container.

Sure implementing a gRPC interface might not be the hardest thing in hindsight, but I've never done it before and it's now a fully functional(ish) implementation of what i described.

It even managed to switch gRPC implementations because the Python one was funky with protoc versions in Nix(Python bundles the grpc codegen it's so stupid) so i asked it to do the same thing for grpclib instead which worked.

In the past you looking around documentation, SO answers etc would have hopefully helped you learn more about the tools and develop skills required to independently analyze the pros and cons. If you ask an LLM (or a search engine or a colleague) and take their words as the ground truth then you won’t develop the skills. Worst, sooner or later no one will have enough knowledge nor analytic skill to form an opinion on any sufficiently deep subject and they will all be dependent on corporate chatbots to spoon feed them information that may or may not be biased in a way against your interest. Now imagine if ChatGPT tells you to use azure and Gemini tells you to use GCP…

Even when using them to code, I use them as a search engine. Rather than telling them to implement feature X, I clone a repo which has a similar feature and say:

"explore the repo I've cloned at /src/foo and explain how it achieves barFeature. Now look a the project at /src/baz and tell me why it would be difficult to use foo's approach in baz"

I rarely have it do anything novel, just translate ideas from existing projects into mine. Novel work is for me to enjoy coding directly.

It is nice when it works, but sometimes I run into trouble where I don't know a right word to put in the prompt to get the answer I'm looking for. I've recently been playing around with Raku and had a really cryptic type signature error and Claude was of absolutely no help because it didn't know about the interaction of Signature literals and 'Slurpy' sigils in method parameters. Only when I learned about it and included the word Slurpy in my prompt would it actually regurgitate the information I was looking for but at that point I already knew it.

I would just add that gen AI can actually be a fantastic aid when you're building some toy projects just for yourself.

I am primarily a backend engineer, though I am capable on the front end, just not particularly fast. I am especially slow at the voodoo that is CSS. In the past, I would have been turned off from doing some end-to-end personal projects because I know I would have been bogged down just battling CSS for hours and hours (it's particularly problematic that I'm slow at CSS but also a stickler for how I want things to look). With AI tools I can literally just say "make it pretty". It gets me about 85% of the way there, but importantly it gets me to the point where it puts in all the styling framework and then I just have to fix bugs/edit/tweak it, which goes very quickly.

So now I've found myself much more likely to build personal projects because I know I can get past things that in the past felt a bit like quicksand to me.

I ended up migrating to fixing things in the libraries I use because they piss me off.

Bad onboarding docs, broken or bent SDLC, major performance issues. I spent all day yesterday fixing perf issues in a library that competes with one I loathe for trying to complect an entire problem domain.

You probably have the same thing I do. Teamwork is a very different motivation than working on something either never intended for others or with a long MVP cycle that makes it too easy to get bogged down with side quests. "It would be way less tedious to deal with X if I did Y first. But now Y is blocked so I do nothing."

Versus 'someone is waiting for this.'

Out of curiosity, how did you end up with 6 months off between jobs?

I would love the opportunity to take ~6 months off, but I feel scared that if I just quit, it might take me a longer time to find a job again.

In my younger years, classic ASP + SQL was easy, and I had IIS + SQL Server set up on a server at my house. And I could write HTML/CSS/JavaScript.

Deployment was... run update scripts, push ASP files via FTP.

Now I know there are newer ways to do things, and I'd love to set up CI/CD in my personal life, but every time I want to do a "toy project" I get hung up researching and thinking about my own personal software development life cycle.

What you choose for hosting and deployment for your toy projects?

I've done the same, but with AI generating a lot of the boilerplate, and helping generate automated tests, has accelerated this process considerably.

Unfortunately the leet coding is key to getting the next job

I actually only like working on software that has a real use, at least to myself. The coolest is when I spin something up fast at work and people start relying on it.

Same goes for my car, it's satisfying doing inexpensive fixes to keep it running. Wouldn't want some mid-2000s BMW project that's stuck in the garage half the time.

I used to self-host my email. I don't any more, for exactly this reason. And of course this applies to every other "just do it yourself!" enthusiast appeal. I actually want to delegate it to someone who does it full time.

I recently wrote my own invoicing application. I got wrapped up in the joy of adding the features I wanted. Many of these features come at a premium monthly fee for a big named product.

I needed to get an invoice out in a timely fashion. As much as I wanted to use my app, I found certain kinks I needed to work out (with styling, adding addresses, etc.) -- This was where I realized what you have articulated.

At some point, it becomes better to prioritize the "fun" in working on my bike, and the "usefulness" of the daily driver bike.

In doing so, perhaps the fun/usefulness of each converges over time.

I also have more "working software" interests than i have time and energy. Many of them have a ton of mundane boring details i've done a thousand times before. I'm actually a bit excited that i can work on what i want to work on.

With that said it's a challenge and a chore on its own to curate what "AI" writes. So it's definitely not all roses.

Disclaimer: I'm still early in the experimentation phase. We'll see if i hold these opinions in a few months.

This is why I love building a personal website. It's like a live playground.

I dunno. Working on a bike you want to use to ride to work could be stressful if it is your only bike, you need it to get to work, and your manager will be really mad if you don’t show up to work one day. If there was any slip available it wouldn’t be so bad.

Actually, I really like this analogy because

1) We can’t really work on non-trivial fixes for our cars because they are too complex

2) We’ve totally structured our living situations around these over-complex beasts

3) Sometimes they do just break down at random, and it becomes a mini-crisis for everyone involved, which can’t really be resolved without missing a day of work to go to a professional

4) The bike is in exactly the same situation, except you can actually fix most bike problems yourself with a little kit

And yet, it is seen as a serious adult thing to have a car, instead of a bicycle.

Some thoughts:

Outside of, like, customer service, most places are throughput-oriented and it doesn’t even really matter all that much if somebody misses a day.

If we really cared about “uptime” of employees, we’d all be living in bicycle range of our jobs, and able to fix our bikes. Maybe there’d be a bus route as well.

The easy fix is to have two bikes. You can take the time to work on the offline bike, not to mention get needed parts.

>Working on the bike you need to ride to work tomorrow is stressful.

How about working on your bike that other people also work on daily? That's work.

Your project is vastly more specified than a toy project, because it has a real customer (you) who will expect it to work after it ships. That expectation is what separates toys from real tools.

I bet you could write a word processor in an hour. It might only add or delete one character at a time (no selections). It might persist only to a hard-coded "output.txt" filename. You might have to kill the process because there's no quit operation. But that would suffice for a toy word processor.

I think that if you interpret "X days" as "24*X hours" this list becomes much more realistic.

For me, one of the enjoyable aspects of toy projects is that there is no schedule to meet. Which means that I can take my time. And yes, that means I take much more time than the article implies. As an examine, I've been polishing my PEG-based Turing complete language (which features both AOT-to-C and a fully featured repl) since COVID was a daily aspect of life.

I mean, all of those depend on how much was imported as third party dependency, how much was not addressed at all, and what was actually solved manually.

They're all doable if you focus on the core problems. He's got a github with many of the examples up, it's instructive on how to keep things small.

And I think it's important that (based on the code) he had a solid understanding of the problem space before starting to write code. You don't write that tight when you explore a new space.

Edit: https://github.com/ssloy?tab=repositories for the shortcut link :)

A lot of people say if you don’t use LLMs then you will fall behind. I’m starting to think that not using them will be a significant advantage in the long run.

> "by the way if you have friends who are expert programmers, I don't recommend asking them for help."

AI is not an expert programmer [today]. and it doesnt take an expert programmer to arrive at that conclusion.

This is by far my most common usage of LLMs, and they’re good at it. Sometimes you have to be intentional about preventing it from being sycophantic and just telling you that you’re right, through a system prompt or by phrasing the question such that it’s comparing ideas neutrally instead of comparing “your” idea to some other idea.

It feels like Claude and ChatGPT have both become more sycophantic over the past months.

(I'm the post author) Actually, I would recommend that you don't ask an expert friend for help. If you get really stuck then maybe do some light reading about the topic: but the point is to throw lots of things at the wall and puzzle your way through it. Figuring things out from first principles is fun and also provides you with a litany of creative thinking skills to help you tackle other problems. I firmly believe that spending time confused is an essential ingredient in the process.

Obviously, you're welcome to do as you please though, mileage may vary, etc.

I try to do exactly that. I phrase it as using LLM as teachers instead of interns.

I think what you've identified is the "certain kinds of learning" I mention in the post.

If I want to learn about the structure of ELF binaries, I don't figure it out from first principles. I can't: the decisions that went into its design where highly specific to its history and the sensibilities of its creators, and in a modern context a lot of those decisions are near enough random. These are the sorts of things you should use specs, docs, Google, and maybe an LLM for.

What I'm talking about is, I guess, 'constructive' learning. I might not be able to build my own ELF parser through guesswork, but I probably can write a binary format of my own and a simple OS kernel to read it, given enough time. It probably won't look much like ELF, but the process of discovery that'll happen will teach me about why ELF came to be and what problems is solving, and also about the wider design space that sits around it.

My suggestion is that you should not use an LLM's help for the latter.

I like to think of it this way: if you bolted me to a chair in a room with nothing but a laptop, a text editor, and a compiler, what might I be able to create if you left me for 10 years? What couldn't I create because I wouldn't have sufficient implementation-specific information?

I struggle with Claude fixing a bug and not giving me the updated output. Like explicitly having to ask it for the fix 6 times while it insists the update is in the output. Did you ever have that happen?

> The problem with that is making a configuration engine is a lot less efficient than just writing the code to do the damn thing.

http://mikehadlow.blogspot.com/2012/05/configuration-complex...

It took me a solid decade to accept these lessons. I cannot build a product that even remotely competes with things like Visual Studio and .NET tooling. How many hours would it take a solo developer to recreate a debugger experience that anyone else could be bothered to look at for more than 5 seconds?

The hardest part of building something configurable is the UI/UX. The moment you step outside of the host language, your tooling support goes to zero. Sure, you can pick other popular languages as the language of configuration (SQL, python, Lua, etc.) and piggyback off those ecosystems. But, you still have to build a way for a "non programmer" to visit all of the patchwork sites of "configuration" in a coherent way. Telling the business people to go edit a folder full of .py and .sql files will probably not be received well. Wrapping all of this with a fancy web interface doesn't make the problem less severe either (it makes it substantially worse).

If you have a way to build/deploy your product in <60 seconds from source, worrying about elaborate configuration and customization architecture seems extremely idiotic to me now. The best path I've found is something approximating a common library and a per-customer executable that consumes it and implements the expected contracts. Each customer's project lives in a subfolder on the same monorepo. The power of the hierarchical filesystem is tragically understated throughout our industry.

A rite of passage! :D

[dead]

This is also a great way to stand out among other candidates in the resume-review stage of a hiring process.

The number one question I have as a hiring manager is always "can this person actually write code". A GitHub profile with 2-3 projects with multiple commits to each one answers that question really well. I won't filter out people who don't have that - lots of great engineers have never published code before - but the people who do have that are much more likely to get picked for the phone screen.

I've been using vhs ( https://github.com/charmbracelet/vhs ) to script gif creation- really showcases CLI workflows if you're demoing multiple commands.

Examples: https://github.com/bbkane

Thanks, I'll try it. I have a few things knocking around.

I love this take, and I’m reminded of the important idea that while we all are encouraged to be productive, our own joy is also an important thing for us to produce.