Skip to content(if available)orjump to list(if available)

Syntax highlighting is a waste of an information channel (2020)

tschumacher

My coworker recently showed me this plugin [1] that fades out all Rust code that is unrelated to the variable under the cursor. Think of it as a more powerful version of the "click to highlight all appearances" you can do in most IDEs but it actually does information flow analysis on the code.

[1]: https://github.com/willcrichton/flowistry

thomascountz

I like using syntax highlighting when it breaks. For example, if all code below a particular line is a single color, then I probably forgot an end-quote or something. But this has become less uniquely useful due to the broader integration of parser-driven linters (e.g. tree-sitter), which—besides being able to drive highlighting—can explicitly deliver inline hints or parsing errors.

All that said, I'm one who appreciates information density! How about coloring branching code paths/call stacks?

My keyboard has a concept of "layers," which allows each key to map differently depending on the layer. I've seen this used to make a numpad or to have a QWERTY and DVORAK layer. What if highlighting was the same? Instead of competing for priority over the color channels, developers could explicitly swap layers?

AceJohnny2

Great point! Similarly, I sometimes use Emacs' excellent (and near-unique) electric-indent as a hint of syntax brokenness. "What do you mean this is getting indented at that lev-- oooh"

The downside with broken syntax highlighting (and electric-indent!) is when the editor's parser is insufficient, as is often the case with basic online editors, and breaks with legitimate constructs (Emacs with certain C macros). Then I can't trust the highlighter and also I have less-legible code.

btreecat

> All that said, I'm one who appreciates information density! How about coloring branching code paths/call stacks? > > My keyboard has a concept of "layers," which allows each key to map differently depending on the layer. I've seen this used to make a numpad or to have a QWERTY and DVORAK layer. What if highlighting was the same? Instead of competing for priority over the color channels, developers could explicitly swap layers?

I was thinking about coloring logic /scope blocks as a way to help visualize scope and flow, even if it required static analysis and a simple script it could be useful when I need to debug

conartist6

For the last five years I've been working on this problem!

To solve it we need to be able to describe the structured content of a document without rendering it, and that means we need an embedding language for code documents.

I hope this doesn't sound overly technical: I'm just borrowing ideas from web browsers. I think of my project as being the creation of a DOM for code documents. The DOM serves a similar function. A semantic HTML documents has meaning independent of its rendered presentation and so it can be rendered many ways.

CSTML is my novel embedding language for code. You could think of it like a safe way to hold or serialize an arbitrary parse tree. Like HTML a CSTML document has "inner text" which this case is the source text if the program the parser saw. E.g. a tiny document might be `<Boolean> 'true' </>`. The parser injects node tags into the source text, creating what is essentially the perfect data stream to feed a syntax highlighter. To do the highlighting you print the string content if the document and use the control tags to decide on color. This is actually already how we syntax highlight the output from our own CLI as it happens. We use our streaming parser technology to parse our log output into a CSTML tag stream (in real time) and then we just swap out open and close node tags for ANSI escape codes, print the strings, and send that stream to stdout.

Here's a more complicated document generated from a real parse: https://gist.github.com/conartist6/412920886d52cb3f4fdcb90e3...

jasonwatkinspdx

I guess I may be an outlier here but I don't find any of those examples motivating at all. They make it harder for me to see the structure of the code at a glance.

Also I think it's a bit off the mark to think of it as being a wasted information channel. Redundancy is a feature of human languages, because our languages are not optimizing solely for density. A bit of redundancy helps our brains pattern match more consistently, almost like a form of forward error correction. Syntax highlighting is like that, at least for me, where it makes a big difference in seeing the structure at a glance, and more overly complex coloring rules thwart that for me. Like I don't want to be trying to match up rainbow shades of parens.

babypuncher

I like rainbow parentheses and this is already a common feature in code editors.

Every example past that was just worse for readability. I think you're right about density not being the only important metric here.

measurablefunc

Syntax highlighting is a type of abstract interpretation¹. The author wants to customize it which makes sense & most text editors do have enough programmability to make that possible as long as you are willing to write your own abstract interpreter. Presumably it should also be possible to take the syntax specification along w/ a description of a language runtime & let an AI agent automatically generate custom abstract interpreters. This is not currently possible but might be in a few years if AI labs focus on technical advancement instead of whatever OpenAI has been focusing on recently w/ media synthesis.

¹https://www.cs.cmu.edu/~emc/15817-s11/lect-2011-03-23.pdf

swyx

(OP here) i posted this old blog actually because i'm working on a similar AI powered feature now. teased a bit here but unreleased https://x.com/swyx/status/1977906767771086898

i feel like HN might be the only place i can explain the "AI syntax highlighting" angle and have somebody get it. the Codemaps UX isnt exactly tuned for the exact form factor of syntax highlighting, but the general idea of "hey you can selectively highlight code based on what you're currently trying to do, and btw also reorganize your filesystem accordingly" is kinda cool and would love ideas on how to best express it.

measurablefunc

Sounds like you are doing reachability analysis. The general principles come from the theory of abstract interpretation. You're trying to map syntax to corresponding elements in an abstract semantic domain but I don't think you can do that w/ existing AI tools b/c none of the tools can generate an abstract interpreter according to a specification & prove its correctness.

stevage

I don't know if it's just because the article is 5 years old, but many of those things do exist in my editor (vs code) with the language extensions and listing I use. Rainbow parentheses, long lines, variables not assigned to, etc etc.

The colour channel is being well and truly used to close to it's maximum.

1718627440

Having this in the editor leads to people removing that information from other channels. For example class properties are now often coloured, so people do not name or refer to them to denote that they are not some random other variable. I don't like this, because I think all information should be in the code itself and it can be really annoying once you work and haven't your favorite editor available in the correct setup.

em-bee

with the same argument i used to reject syntax highlighting in general. all the information should already be in the code. if i can't read that, then the syntax of the language is bad. syntax highlighting would just make me lazy. i have since decided that i like being lazy so i do like to use it, but you make a good point, if the use of colors makes you not write something in the code that you would otherwise, then that's not good. i can see that with syntax highlighting too, where the colors help me read code even if it is badly formatted, so i am less inclined to format for readability.

1718627440

True, but I haven't experienced that for general syntax highlighting. I do read the code through cat or in other contexts often enough that I haven't lost the ability to read code at normal speed without syntax highlighting. Do you have an example?

em-bee

i wasn't talking about the ability to read code, even though that was my initial argument for rejecting colors. what i mean is that without colors i might take more effort to write/format code so that it is more readable, instead of relying on syntax highlighting:

with colors this is perfectly readable, because if, for and return appear in red, and other keywords in blue. so they stand out, making the structure more visible than without colors.

    mixed reduce(function fun, array arr, mixed|void zero) {
      if(sizeof(arr))
        zero = arr[0];
      for(int i=1; i<sizeof(arr); i++)
        zero = ([function(mixed,mixed:mixed)]fun)(zero, arr[i]);
      return zero; }
without colors i might prefer to write something like this. using braces around each block, and line breaks, to make each part stand out.

    mixed reduce(function fun, array arr, mixed|void zero)
    {
      if(sizeof(arr))
      {  
        zero = arr[0];
      }

      for(int i=1; i<sizeof(arr); i++)
      {
        zero = ([function(mixed,mixed:mixed)]fun)(zero, arr[i]);
      }

      return zero;
    }
without colors clearly the second is easier to read.

github uses a different color scheme but maybe you can get the idea:

https://github.com/pikelang/Pike/blob/fe4b7ef78cc26316e62e79...

mouse_

Rainbow parentheses blew my mind. Why isn't every editor already doing this?

crazygringo

Because it just turns into rainbow noise.

What editors like VSCode do instead is to highlight the matching parenthesis when the cursor is over the other. That way you don't have to hunt for a matching color.

And to make any unmatched opening parenthesis bright red. The invalid state is made very clear.

Together, these are much more effective.

recursivecaveat

I have tried it before, unfortunately never found it that handy. For starters you need a pretty complicated expression not already disambiguated by layout to start to care. Which is the sort of thing people try to avoid anyways. Second, there are not a huge number of colors that are super distinct when not right up against each other, doubly so when you consider all of them must be very distinct from the background color. Lastly there's a lot of finickiness with other usages of colours conflicting.

I would actually prefer the opposite. Render the () characters as different matching glyph pairs. The space for distinctive asymmetric glyphs is a lot larger and not generally very loaded because people code 99.9% in ascii.

AndrewDucker

VSCode does it, and it makes it very easy to both snip out a scope and to tell when you've got the wrong number of closing brackets.

1718627440

The built-in editor of Casio calculators is doing this, on some crappy 80x8 screen. I think this is not too uncommon. My editor only colours the parentheses that I currently edit, which I find much less intrusive and has the same (I would say a better) effect.

navane

Doesn't excel do this? Arguably the most used ide!

denimnerd42

rainbow spaces/tabs too

throawayonthe

most have an extension or setting to do this already!

mlukaszek

Terrible idea for color-blind people. And I don't even mean severely color-blind, like when you can't distiguish red from green at all, but something more common and less severe, like deuteranomaly - where it's shades of these colors that are hard to distinguish.

shakna

Because the rainbow parenthesis alternate in frequency of brightness, its actually really easy to distinguish which is which, unless you're severely colour blind.

I can't tell some apart, but because they've got a different colour in-between, it makes it easier to jump between start and end of expressions. Being able to box blocks in my head faster.

That being said, I always have to tweak accessibility settings anyway. Change of font, change of size. Having to toggle off rainbow as well doesn't seem to really add to the large list of things.

jampekka

The color blind would still see the parenthesis at least like they see them currently without the color coding though, given they are distinguishable from the background by lightness.

null

[deleted]

watwut

Those people are able to distinguish suggested colors - the example is using primary colors. And at worst, it would be the same as all parenthesis being the same color (black).

convolvatron

no, its not. first of all either I sense brightness differently, or I use it to compensate. so highlighting of any kind make the text a mix of brightnesses from dingy to glaring across the text.

since it take me effort to actually parse the colors, this is a constant distraction.

so I can read monochrome text just fine, but multi-colored text really slows me down.

jryb

Many (most?) of these are achievable with neovim and tree-sitter (a plugin that gives you access to the AST), and surely many other editors. I have plugins installed right now that do several of the things that are mocked up here. Many more are done with virtual text and not color, but I don’t see why you couldn’t use highlighting instead.

I agree with the broader point of the article that color is underused, but the state of the art has moved way past what the author’s tools are currently configured to provide.

jasonjmcghee

To be fair, the article is over 5 years old.

The author seemed to be unfamiliar with tree-sitter (first appeared in 2018) and incorrectly assumed Atom used TextMate.

Since then it's gotten much more popular and adopted by other editors.

jryb

Good catch, thanks!

stickfigure

I like the rainbow parens, but I feel like many of these are done already by IntelliJ. And not with just fonts and colors, but with little inline annotations, bubbles, and squiggly lines.

One thing I am still asking for: I want to be able to clearly and obviously distinguish mutable state from immutable state. IntelliJ can't do it yet.

NalNezumi

Reminds me of my college note taking practice. I used to take notes with about 3-5 different color pens, and my friends used to be puzzled about it (it sure looks weird to swap between pens often).

I used to reply that the color pens made it easier to keep context such as what teacher said was important, what I found difficult, when in the note I had an "aha!" moment, side comment from me, Q&A asked by student during lecture, or how certain things written down now is related to the point made earlier/later in the lecture/notes.

Text (note) is the content but our (at least mine) attention are not really made for plain text. There's so much more you can play with visual information.

andai

Tangential but you are the exact target audience for those pens that were popular in the 2000s, where it was 4 colored ball points combined into one pen.

jasonwatkinspdx

Those things were all the rage even back in elementary school in the 80s.

seanw444

Those were so great.

skirmish

I still remember my math undergrad notes from years ago: definitions had a prefix Def circled red, theorems -- Th in blue, examples -- Ex in green. Made it much easier to review my notes before midterms / exams.

_boffin_

Whenever I jump into excel or any spreadsheet app, light light blue is user input and light light red / pink / salmon is final formula output. Makes it so much easier.