Differentiable Logic Cellular Automata
97 comments
·March 6, 2025bob1029
fnordpiglet
Yeah this paper feels profoundly important to me. The ability to differentiate automata means you can do backward propagating optimization on Boolean circuit designs to learn complex discrete system behaviors. That’s phenomenal.
satvikpendem
What a busy beaver you are.
throwaway13337
This is exciting.
Michael Levin best posited for me the question of how animal cells can act cooperatively without a hierarchy. He has some biological experiments showing, for example, eye cells in a frog embryo will move to where the eye should go even if you pull it away. The question I don't think he could really answer was 'how do the cells know when to stop?'
Understanding non-hierarchical organization is key to understanding how society works, too. And to solve the various prisioner's delimmas at various scales in our self-organizing world.
It's also about understanding bare complexity and modeling it.
This is the first time I've seen the ability to model this stuff.
So many directions to go from here. Just wow.
fc417fc802
> The question I don't think he could really answer was 'how do the cells know when to stop?'
I'm likely missing something obvious but I'll ask anyway out of curiosity. How is this not handled by the well understood chemical gradient mechanisms covered in introductory texts on this topic? Essentially cells orient themselves within multiple overlapping chemical gradients. Those gradients are constructed iteratively, exhibiting increasingly complex spatial behavior at each iteration.
cdetrio
Textbook models typically simulate normal development of an embryo, e.g. A-P and D-V (anterior-posterior and dorsal-ventral) patterning. The question Levin raises is how a perturbed embryo manages to develop normally, both "picasso tadpoles" where a scrambled face will re-organize into a normal face, and tadpoles with eyes transplanted to their tails, where an optic nerve forms across from the tail to the brain and a functional eye develops.
I haven't thoroughly read all of Levin's papers, so I'm not sure to what extent they specifically address the issue of whether textbook models of morphogen gradients can or cannot account for these experiments. I'd guess that it is difficult to say conclusively. You might have to use one of the software packages for simulating multi-cellular development, regulatory logic, and morphogen gradients/diffusion, if you wanted to argue either "the textbook model can generate this behavior" or that the textbook model cannot.
The simulations/models that I'm familiar with are quite basic, relative to actual biology, e.g. models of drosophila eve stripes are based on a few dozen genes or less. But iiuc, our understanding of larval development and patterning of C Elegans is far behind that of drosophila (the fly embryo starts as a syncytium, unlike worms and vertebrates, which makes fly segmentation easier to follow). I haven't read about Xenopus (the frogs that Levin studies), but I'd guess that we are very far from being able to simulate all the way from embryo to facial development in the normal case, let alone the abnormal picasso and "eye on tail" tadpoles.
triclops200
I'm not an expert on the actual biological mechanisms, but, it makes intuitive sense to me that both of those effects would occur in the situation you described from simple cells working on gradients: I was one of the authors on this paper during my undergrad[1] and the generalized idea of an eye being placed on a tail and having nerves routed successfully through the body via pheromone gradient is exactly the kind of error I watched occur a dozen times while collecting the population error statistics for this paper. Same thing with the kind of error of a face re-arranging itself. The "ants" in this paper have no communication except chemical gradients similar to the ones talked about with morphogen gradients. I'm not claiming it's a proof of it working that way, ofc, but, even simpler versions of the same mechanism can result in the same kind of behavior and error.
[1]: https://direct.mit.edu/isal/proceedings/alif2016/28/100/9940...
Jerrrrrry
What are Cognitive Light Cones? (Michael Levin Interview)
calebm
I love playing around with cellular automata for doing art. It's amazing what kind of patterns can emerge (example: https://gods.art/math_videos/hex_func27l_21.html). I may have to try to play with these DLCA.
j_bum
Lovely! Thanks for sharing. Would these patterns keep generating indefinitely?
SomeHacker44
Reminds me of the old movie Andromeda Strain.
EMIRELADERO
I've been thinking a lot about "intelligence" lately, and I feel like we're at a decisive point in figuring out (or at least greatly advance our understanding of) how it "works". It seems to me that intelligence is an emergent natural behavior, not much different than classical Newtonian mechanics or electricity. It all seems to boil down to simple rules in the end.
What if everything non-discrete about the brain is just "infrastructure"? Just supporting the fundamentally simple yet important core processes that do the actual work? What if it all boils down to logic gates and electrical signals, all the way down?
Interesting times ahead.
ekez
There’s something compelling about these, especially w.r.t. their ability to generalize. But what is the vision here? What might these be able to do in the future? Or even philosophically speaking, what do these teach us about the world? We know a 1D cellular automata is Turing equivalent, so, at least from one perspective, NCA/these aren’t terribly suprising.
data-ottawa
Potentially it would be useful if you could enter a grid from satelite images and simulate wildfire spread or pollution spread or similar problems.
achille
these are going to be the dominant lifeforms on earth exceeding bacteria, plants and humans in terms of energy consumption
cellular automata that interact with their environment, ones that interact with low level systems and high level institutions. to some approximation we, humans are just individual cells interacting in these networks. the future of intelligence aint llms, but systems of automata with metabolic aspects. automata that co-evolve, consume energy and produce value. ones that compete, ones that model each other.
we're not being replaced, we're just participants in a transformation where boundaries between technological and cellular systems blur and eventually dissolve. i'm very thankful to be here to witness it
ryukoposting
I'll have what this guy is smoking. Those visualizations are pretty, though.
I can imagine this being useful for implementing classifiers and little baby GenAI-adjacent tech on an extremely tiny scale, on the order of several hundred or several thousand transistors.
Example: right now, a lot of the leading-edge biosensors have to pull data from their PPG/ECG/etc chips and run it through big fp32 matrices to get heart rate. That's hideously inefficient when you consider that your data is usually coming in as an int16 and resolution any better than 1bpm isn't necessary. But, fp32 is what the MCU can do in hardware so it's what you gotta do. Training one of these things to take incoming int16 data and spit out a heart rate could reduce the software complexity and cost of development for those products by several orders of magnitude, assuming someone like Maxim could shove it into their existing COTS biosensor chips.
achille
yes absolutely: current systems are wildly inefficient. the future is one of extreme energy efficiency.
re smoking: sorry let me clarify my statement. these things will be the dominant life forms on earth in terms of metabolism, exceeding the energy consumption of biological systems, over 1k petawatt hours per year, dwarfing everything else
the lines betwen us may blur metaphorically, we'll be connected to them how we're connected to ecosystems of plants and bacteria. these systems will join and merge in the same way we've merged with smartphones -- but on a much deeper level
suddenlybananas
So grandiose. It's a good thing to rapture is happening when you're alive to see it. You're just that important.
achille
i wasn't around to see the first humans land on the moon. i feel a similar deep sense of awe and excitement to see this revolution
ysofunny
because the goal of life is to maximize metabolic throughput?
or to minimze energetic waste?
emmelaich
The self-healing properties suggest biological evolution to me.
spyder
Hmm.. could this be used for the ARC-AGI challenge? Maybe even combine with this recent one: https://news.ycombinator.com/item?id=43259182
eyvindn
:)
JFuzz
This is wild. Long time lurker here, avid modeling and simulation user-I feel like there’s some serious potential here to help provide more insight into “emergent behavior” in complex agent behavior models. I’d love to see this applied to models like a predator/prey model, and other “simple” models that generate complex “emergent” outcomes but on massive scales… I’m definitely keeping tabs on this work!
marmakoide
Self-plug here, but very related => Robustness and the Halting Problem for Multicellular Artificial Ontogeny (2011)
Cellular automata where the update rule is a perceptron coupled with a isotropic diffusion. The weights of the neural network are optimized so that the cellular automata can draw a picture, with self-healing (ie. rebuild the picture when perturbed).
Back then, auto-differentiation was not as accessible as it is now, so the weights where optimized with an Evolution Strategy. Of course, using gradient descent is likely to be way better.
emmelaich
The result checkerboard pattern is the opposite (the NOT) of the target pattern. But this is not remarked upon. Is it too unimportant to mention or did I miss something?
eyvindn
thanks for catching this, the figure for the target was inverted when exporting for publication, corrected now.
vessenes
Amazing paper, I re-read it in more detail today. It feels very rich, like almost a new field of study —- congratulations to the authors.
I’m ninjaing in here to ask a q — you point out in the checkerboard initial discussion that the 5(!) circuit game of life implementation shows bottom left to top right bias — very intriguing.
However, when you show larger versions of the circuit, and in all future demonstrations, the animations are top left to bottom right. Is this because you trained a different circuit, and it had a different bias, or because you forgot and rotated them differently, or some other reason? Either way, I’d recommend you at least mention it in the later sections (or rotate the graphs if that aligns with the science) since you rightly called it out in the first instance.
miottp
Author here. Thank you! You're seeing that correctly. The directional bias is the result of some initial symmetry breaking and likely random-seed dependent. The version that constructs the checkerboard from the top-right down was trained asynchronously, and the one from the bottom-left up was trained synchronously. The resulting circuits are different.
itishappy
They're learning features, not the exact image (that's why it's so good at self healing). It should be invariant to shifts.
justinnk
This is very interesting! I think an exciting direction would be to arrive at minimal circuits that are to some extent comprehensible by humans. Now, this might not be possible for every system, but certainly the rules of Conway‘s GoL can be expressed in less than 350 logic gates per cell?
This also reminds me of using Hopfield networks to store images. Seems like Hopfield networks are a special case of this where the activation function of each cell is a simple sum, but I’m not sure. Another difference is that Hopfield networks are fully connected, so the neighborhood is the entire world, i.e., they are local in time but not local in space. Maybe someone can clarify this further?
Cladode
Continuous relaxation of boolean algebra is an old idea with much literature. Circuit synthesis is a really well-researched field, with an annual conference and competition [1]. Google won the competition 2 years ago. I wonder if you have tried your learner against the IWLS competition data sets. That would calibrate the performance of your approach. If not, why not?
srcreigh
The Conway's game of life example isn't so impressive. The network isn't really reverse engineering rules, it's being trained on data that is equivalent to the rules. It's sort of like teaching + by giving it 400 data points triplets (a,b,c) with 1 <= a,b <= 20 and c = a + b.
andrewflnr
It wasn't meant to be much more than a sanity check, as I read it anyway.
This is very interesting. I've been chasing novel universal Turing machine substrates. Collecting them like Pokémon for genetic programming experiments. I've played around with CAs before - rule 30/110/etc. - but this is a much more compelling take. I never thought to model the kernel like a digital logic circuit.
The constraints of boolean logic, gates and circuits seem to create an interesting grain to build the fitness landscape with. The resulting parameters can be directly transformed to hardware implementations or passed through additional phases of optimization and then compiled into trivial programs. This seems better than dealing with magic floating points in the billion parameter black boxes.