Karpathy on DeepSeek-OCR paper: Are pixels better inputs to LLMs than text?
14 comments
·October 21, 2025sabareesh
It might be that our current tokenization is inefficient compared to how well image pipeline does. Language already does lot of compression but there might be even better way to represent it in latent space
ACCount37
People in the industry know that tokenizers suck and there's room to do better. But actually doing it better? At scale? Now that's hard.
typpilol
It will require like 20x the compute
CuriouslyC
Image models use "larger" tokens. You can get this effect with text tokens if you use a larger token dictionary and generate common n-gram tokens, but the current LLM architecture isn't friendly to large output distributions.
hbarka
Chinese writing is logographic. Could this be giving Chinese developers a better intuition for pixels as input rather than text?
varispeed
Text is linear, whereas image is parallel. I mean when people often read they don't scan text from left to right (or different direction, depending on language), but rather read the text all at once or non-linearly. Like first lock on keywords and then read adjacent words to get meaning, often even skipping some filler sentences unconsciously.
Sequential reading of text is very inefficient.
sosodev
LLMs don't "read" text sequentially, right?
olliepro
The causal masking means future tokens don’t affect previous tokens embeddings as they evolve throughout the model, but all tokens a processed in parallel… so, yes and no. See this previous HN post (https://news.ycombinator.com/item?id=45644328) about how bidirectional encoders are similar to diffusion’s non-linear way of generating text. Vision transformers use bidirectional encoding b/c of the non-causal nature of image pixels.
yunwal
> The more interesting part for me (esp as a computer vision at heart who is temporarily masquerading as a natural language person) is whether pixels are better inputs to LLMs than text. Whether text tokens are wasteful and just terrible, at the input.
> Maybe it makes more sense that all inputs to LLMs should only ever be images.
So, what, every time I want to ask an LLM a question I paint a picture? I mean at that point why not just say "all input to LLMs should be embeddings"?
fspeech
If you can read your input on your screen your computer apparently knows how to convert your texts to images.
CuriouslyC
All inputs being embeddings can work if you have embedding like Matryoshka, the hard part is adaptively selecting the embedding size for a given datum.
smegma2
No? He’s talking about rendered text
rhdunn
From the post he's referring to text input as well:
> Maybe it makes more sense that all inputs to LLMs should only ever be images. Even if you happen to have pure text input, maybe you'd prefer to render it and then feed that in:
Italicized emphasis mine.
So he's suggesting that/wondering if the vision model should be the only input to the LLM and have that read the text. So there would be a rasterization step on the text input to generate the image.
Thus, you don't need to draw a picture but generate a raster of the text to feed it to the vision model.
Recent and related:
Getting DeepSeek-OCR working on an Nvidia Spark via brute force with Claude Code - https://news.ycombinator.com/item?id=45646559 - Oct 2025 (43 comments)
DeepSeek OCR - https://news.ycombinator.com/item?id=45640594 - Oct 2025 (238 comments)