How I keep up with AI progress

I strongly agree with this sentiment and found the blog's list of "high signal" to be more a list of "self-promoting" (some good people who I've interacted with a fair bit on there, but that list is more 'buzz' than insight).

I also have not experienced the post's claim that: "Generative AI has been the fastest moving technology I have seen in my lifetime." I can't speak for the author, but I've been in this field from when "SVMs are the new hotness and neural networks are a joke!" to the entire explosion of deep learning, and insane number of DL frameworks around the 20-teens, all within a decade (remember implementing restricted Boltzmann machines and pre-training?). Similarly I saw "don't use JS for anything other than enhancing the UX" to single page webapps being the standard in the same timeframe.

Unless someone's aim is to be on that list of "High signal" people, it's far better to just keep your head down until you actually need these solutions. As an example, I left webdev work around the time of backbone.js, one of the first attempts at front end MVC for single pages apps. Then the great React/Angular wars began, and I just ignored it. A decade later I was working with a webdev team and learned React in a few days, very glad I did not stress about "keeping up" during the period of non-stop changing. Another example is just 5 years ago everyone was trying to learn how to implement LSTMs from scratch... only to have that model essentially become obsolete with the rise of transformers.

Multiple times over my career I've learned lesson that moving fast is another way of saying immature. One would find more success learning about the GLM (or god forbid understanding to identify survival analysis problems) and all of it's still under appreciated uses for day-to-day problem solving (old does not imply obsolete) than learning the "prompt hack of the week".

Beyond a basic understanding of how LLMs work, I find most LLM news fits into one of these categories:

- Someone made a slightly different tool for using LLMs (may or may not be useful depending on whether existing tools meet your needs)

- Someone made a model that is incrementally better at something, beating the previous state-of-the-art by a few % points on one benchmark or another (interesting to keep an eye on, but remember that this happens all the time and this new model will be outdated in a few months - probably no one will care about Kimi-K2 or GPT 4.1 by next January)

I think most people can comfortably ignore that kind of news and it wouldn’t matter.

On the other hand, some LLM news is:

- Someone figured out how to give a model entirely new capabilities.

Examples: RL and chain of thought. Coding agents that actually sort of work now. Computer Use. True end-to-end multimodal modals. Intelligent tool use.

Most people probably should be paying attention to those developments (and trying to look forward to what’s coming next). But the big capability leaps are rare and exciting enough that a cursory skim of HN posts with >500 points should keep you up-to-date.

I’d argue that, as with other tech skills, the best way to develop your understanding of LLMs and their capabilities is not through blogs or videos etc. It’s to build something. Experience for yourself what the tools are capable of, what does and doesn’t work, what is directly useful to your own work, etc.

When explaining LLMs to people, often the high level architecture is what they find the most interesting. Not the transformer, but the token by token prediction strategy (autoregression), and not always choosing the most likely token, but a token proportional to its likelihood.

The minutiae of how next token prediction works is rarely appreciated by lay people. They don't care about dot products, or embeddings, or any of it. There's basically no advantage to explaining how that part works since most people won't understand, retain, or appreciate it.

Indeed. We have just had a few really big shifts since launch of GPT3. Rest has just been bigger and more optimized models + tooling around the models.

AI research was so interesting pre-transformers it was starting to get a bit wild around GPT2 IIRC but now the signal to noise is so low with every internet sensationalist and dumb MBA jumping on the bandwagon.

It's not a model of text, though. It's a model of multiple types of data. Pretty much all modern models are multimodal.

I agree, but with the caveat that it's probably a bad time to fall asleep at the wheel. I'm very much a "nothing ever happens" kind of guy, but I see a lot of people who aren't taking the time to actually understand how LLMs work, and I think that's a huge mistake.

Last week I showed some colleagues how to do some basic things with Claude Code and they were like "wow, I didn't even know this existed". Bro, what are you even doing.

There is definitely a lot of hype and the lunatics on Linkedin are having a blast, but to put it mildly I don't think it's a bad investment to experiment a bit with what's possible with the SOTA.

Exactly. I have been questioning the need to "keep up". As the best AI innovations arrive, the ones that are useful will eventually make their way to the mainstream (and me). I didn't "keep up" with the nascent development of spreadsheets or Google Docs. Once they were widely used, I adopted them just fine, and haven't missed out on anything.

Unless you are building an AI startup furiously searching for PMF before your runway expires, I don't see the urgency.

Trying to keep up is like jumping on a 90mph treadmill. I have decided to opt out. I think AI (and currently LLMs) is more than a fad and not going away but it's in a huge state of churn right now. I'm not investing a ton of time into anything until I have to. In another few years the landscape will hopefully be more clear. Or not, but at least I won't have spent a lot of time on stuff that has quickly become irrelevant.

I'm currently not using AI or LLMs in any of my day-to-day work.

Yeah these companies have made unbelievable investments, keeping their products a secret is antithetical to their existence.

This is my attitude for all tech surrounding IT

^ https://simonwillison.net

And thankfully, he has an RSS!

Yeah. Time to do something else.

Indeed just get out of tech and make a new living! Tech jobs are declining and will continue then fall off a cliff with one doing the job ten use to. Followed by other white collar and blue collar (AMazons warehouse robots) jobs.

Happily canceled my GPT Plus this week; personally not gonna feed that beast any longer! As well it can not generate maps (create road trip travel maps showing distance between locations to share with friends, a creek tubing map route & etc) at all like Gemini can for free.

Too lazy for that, a Fireship accompanying the morning coffee must suffice.

+1 - super high quality and almost no clickbait