Caching is an abstraction, not an optimization
75 comments
·July 1, 2025ckdot2
bloppe
"Two programs could have similar behaviour but structured very differently, the difference being that one utilizes caching as an abstraction and one explicitly has the concept of different tiers of storage."
The author is comparing "off-the-shelf" caching with custom caching. They're coming from the assumption that you must be caching somehow and arguing that the word "caching" should be understood to mean only particular approaches to the general idea of caching. And obviously the whole point of the general idea is to optimize things.
It's a rhetorical mess
AdieuToLogic
> There's that famous quote "There are only two hard things in Computer Science: cache invalidation and naming things.", and, sure, it's a bit ironical, but there's some truth in there.
The joke form of this quote goes along the lines of:
There are only two hard things in Computer Science: cache
invalidation, naming things, and off-by-one errors.
heikkilevanto
Caching is simple, yes. The hard part is in the last word, invalidation. Even that is manageable for a single process. But as soon as you have multiple (threads / processes / nodes / data centers) updating the data, it does get quite complex, pretty fast.
Likewise, naming things is simple as long as you alone, or a in a small team. But as soon as there are multiple organizations with all their own traditions, it gets tricky. Just witness the eternal flame wars about camelCase, PascalCase, snake_case, kebab-case, and UPPER_CASE. It is almost as hopeless culture clash as Emacs vs Vi vs PowerPoint...
(I leave the off-by-one errors as an exercise for the reader)
TeMPOraL
I'd say this is not the "naming things" that's hard. Beyond picking a common identifier format in the team, there are at least two dimensions that are much harder:
- The language dimension - choice of words, that are good enough for the purpose, and not confusing. For example, "Manager" is as ambiguous as it gets, it can mean many thing, except we've been using it long enough that there's a more specific shape of meaning[0] for that word in code/program architecture contexts - so you still would use it instead of, say "Coordinator", which would raise all kinds of questions that "Manager" no longer does.
- The epistemological dimension - whether the word you chose correctly names the concept you meant, and whether the concept you meant is actually the right one to describe the thing you're trying to describe. Ultimately, this is the hard thing at the root of philosophy. In practice, it manifests like e.g. choice between digging into some obscure branches of mathematics to correctly name the thing "endofunctor" or something, or calling it "Square" and saying "fuck it, we'll clarify the exceptions in the comments".
--
[0] - I mean "more specific" in the sense it's distinct from the other meanings and somewhat narrow - but still it's fuzzy as heck and you can't describe it fully in words; it's basically tacit knowledge.
Xss3
I try to name things descriptively in simple terms and often end up with NamesAboutThisLong, once they get too long i know the thing is doing too much and some refactoring is needed for readability.
I also avoid letting the reader make assumptions. HasPlayerJumpedRecently is bad. What does recently mean? HasPlayerJumpedInLastTenMs is better, even if it's a bit long...Which highlights that it should probably be refactored into a more flexible value; MsSincePlayerLastJumped.
If you arent assuming a time var wth Ms is milliseconds you aren't doing games dev so that one slides with me.
gblargg
I figured the naming issue is deciding how much context. A name might begin inside an organization but need to endure a wider area. If you make all names so long and context-free that they can work in any context, they become unwieldy. Also it can be hard to realize some of the implicit context and what needs to be differentiated with the name. Where server used to suffice, now you need server-a and server-b.
hatthew
If you have a system with "slow storage", caching is a way to optimize that to "storage that is sometimes fast".
If you have a system with "slow storage" and "fast storage", caching is a way to abstract that away to just "storage".
The author is arguing that the latter is the default way we should think about the concept of caching, which is a valid opinion to have.
null
bell-cot
(You forgot off-by-1 errors.)
All software has to name things, and count. Caching (including invalidation) is best understood as a liability. If you can foist it off on your CPU and OS and DB, good for you. Programming whatever you're actually trying to get done is already hard enough.
yxhuvud
Off by 1-errors is not part of the original quote, but is just a later addon to make it funny.
They also tend not to be very hard.
tombert
They're not hard but I will say that when I was writing an app that was using both JavaScript and Julia, I kept getting off-by-one errors because Julia starts at 1 instead of 0.
Really the only time in my entire professional career that off-by-one errors have actually given me headaches.
TeMPOraL
Except when they're part of some base assumptions in the domain or dozen of layers of abstractions below you. They are hard to prevent from happening.
null
null
necovek
On top of the other things mentioned (caching always introduces complexity with lifetime tracking, and thus can't make things simple), the article's got it the wrong way around.
When code has abstract interfaces for data access, introducing caching can be simpler (but not simple) by localizing it in the abstraction implementation which has or doesn't have caching.
But it is not an abstraction (you can perfectly well do caching without any abstractions, and it's frequently done exactly that way).
movpasd
I think you and the article are referring to abstractions over different concerns.
The concern you're talking about is about the actual access to the data. My understanding of the article is that it's about how caching algorithms can abstract the concern of minimising retrieval cost.
So in some ways you're coming at it from opposite directions. You're talking about a prior of "disk by default" and saying that a good abstraction lets you insert cache layers above that, whereas for the author the base case is "manually managing the layers of storage".
necovek
The language used is seriously confusing here.
Algorithms can't really abstract anything since they are, well, just algorithms (formal descriptions of how a computation should be done).
Looking at the author's examples again, I think most everybody would say that caching is used in both:
if data_id in fast_storage:
return fast_storage.get(data_id)
else:
data = slow_storage.get(data_id)
fast_storage.set(data_id, data)
return data
# Uses fast storage or slow storage just like above, but behind the get() method.
return storage.get(data_id)
While there are generic implementations of caching algorithms and we can consider those abstractions, "caching" is a wider term than those implementations, and is specifically not an abstraction (the fact that there is a caching implementation that abstracts something does not make all caching an abstraction).
Edit: Let me also point out that "abstract the concern of minimising retrieval cost" is not caching — I can say that eg. a simple interface with method FastGet(id) does the former, and it needs not use any caching if the underlying structure is fast enough and eg. directly in memory.
foldU
This is correct, I appreciate you for putting it so coherently :). I think I didn’t make it clear enough in the piece that I’m coming from a stance of fast access being table stakes, and the question being about how that’s accomplished.
necovek
"Caching" is an idea of storing a result of an expensive computation in storage that is faster to get from than doing the original computation (in very generic computer terms, computation can be simply fetching from the network or slower local storage).
What you describe as "caching algorithms" are not really caching algorithms, but cached object lifetime management algorithms (LRU, LFU...).
"Abstraction" is a higher level, simplified view of a set of concepts, yet caching is a single concept. See eg. https://en.wikipedia.org/wiki/Abstraction_(computer_science)
It sounds like you are both trying to redefine what "caching" means (tying it to implementations of particular algorithms), but also what "abstraction" means.
We should be very deliberate with the language we use, and our main goal should be to make it simpler to understand, not harder — I believe you are doing the latter here.
armchairhacker
Most optimizations require you to think about how your code is structured, so as a side-effect you make the code more understandable.
In this article, it's cache levels forcing you to separate different types of data because they're accessed at different frequencies. Another example is Rust's borrow checker, whose main purpose is arguably to facilitate both safe and efficient memory management, but which can also be used to enforce invariants that aren't clearly memory-related (e.g. builder pattern, temp files that auto-delete after they're dropped).
These aren't abstractions though. An abstraction is the opposite, hiding structure when it's noisy and making it easier to change. For example, if you already have an architecture in mind and don't want to manually determine how frequently each type of data is accessed, it's better to use a compiler or library that automatically determines what to cache with little to no code or thought on your end; that's abstraction. Similarly, the abstract analogue to Rust's borrow checker is garbage collection, which allows programmers to not think about their data-structures' lifetimes at all. The cost is usually performance and you understand your application less in some ways (although you understand it more in other ways; abstraction hides details but too many details make it hard to see the big picture. Ideally, with abstractions in the right places, you hide only the "unimportant" details in ways that insignificantly affect performance).
klabb3
Note: the author means that caching can be used as an implementation detail in an (abstracted) storage access system, as opposed to a baseline of having multiple storage systems (fast, medium, slow) and managing them directly.
This was confusing to me – the most obvious way to judge the purpose of a system is to compare with the baseline of not having that system at all, especially in the case of caching where the program is functionally complete and correct without a cache. Anyway, there may not be a right or wrong here. Just tripped me up.
LudwigNagasena
Caching is an optimisation. Sometimes caching can be abstracted away, eg CPU cache or build cache are pretty much abstracted away for a usual web developer. But web page caching is very hard to abstract without any abstraction leaks and weird bugs. And even CPU cache is no longer an abstraction if you deal with very high performance code.
gblargg
It sounds like they are arguing that when performance matters, you have to know more about caching. Fair enough, you have to know a lot more about things when optimizing. For a lot of cases you can ignore caching because it can be done transparently. You depend on it to some extent because if e.g. every instruction had to be fetched off rotating storage like the old days, it would play a big role in your design. It's just something solved in general for most software to not have to know much about it.
neuroelectron
This is basically semantic argument, and I will not be engaging in it
jxjnskkzxxhx
You're right, but caching is an optimization.
zmj
This article is talking about single-writer, single-reader storage. I think it's correct in that context. Most of the hairy problems with caches don't come up until you're multi-writer, multi-reader.
gmuslera
"fast storage" is about performance, your abstraction includes performance elements. If you go that down, then you are optimizing on your abstraction designs. What doesn't have to be wrong, but then don't say that is not optimization.
TristanDaCunha
This whole discussion on caching and abstraction was completely befuddling to me.
Joker_vD
There is also an important (but often overlooked) detail that you/your application may not be the only user of the cache. At which point caching, indeed, is an optimization via abstraction: when you fetch an X, you are in no position to predict that the next fifty completely unrelated to you requests would also want to fetch the same X, so it should probably be cached to be readily served.
Which is why solving the "I want my data in fast storage as often as possible" problem may be counter-productive on the whole: you ain't the only client of the system; let it breath and server requests from others.
taeric
This reminds me of the use of materialized views as both a cache strategy and as an abstraction helper.
bravesoul2
And they too can slow things down. Like all caches can. Like Redis can. Cache is a leaky abstraction.
(Although a materialised view is more like an index than a cache. The view won't expire requiring you to rebuild.)
eigenform
Even more obvious if you think about the case of hardware-managed caches! The ISA typically exposes some simple cache control instructions (and I guess non-temporal loads/stores?), but apart from that, the actual choice of storage location is abstracted away from you (and your compiler).
"I think now caching is probably best understood as a tool for making software simpler" - that's cute. Caching might be beneficial for many cases, but if it doesn't do one thing then this is simplifying software. There's that famous quote "There are only two hard things in Computer Science: cache invalidation and naming things.", and, sure, it's a bit ironical, but there's some truth in there.