Levels of configuration languages

A top-level /AppleInternal/ directory on macOS, even if empty, will enable certain features in Apple developer tools.

A number of traditional unix utilities change their behavior based on what their name is. /bin/test and /bin/[ come to mind. but I just checked and a quick survey of openbsd finds.

    eject mt
    [ test
    chgrp chmod
    cksum md5 sha1 sha256 sha512
    cpio pax tar
    ksh rksh sh

And I am guilty of abusing symbolic links as a simple single value key value store. It turns out the link does not need to point to anything and using readlink(1) was easier than parsing a file.

This gives me an idea - store small integer parameters (<= 511) as file permissions (r/w/exe for user/group/other) on an empty file.

I use this approach for testing conditional logic in shell scripts sometimes.

That's just cramming a flag into the file system.

I'm very surprised we don't see more people using a level 5 language to generate Terraform (as level 3 JSON) for this exact reason. It would seem to be the best of both worlds -- use the powerful language to enforce consistency and correctness while still being able to read and diff the simple output to gain understanding. In this hypothetical workflow, Terraform constructs like variables and modules would not be used; they would be replaced by their counterparts in the level 5 language.

https://developer.hashicorp.com/terraform/language/syntax/js...

The problem with HCL is that it's a Level 4 language masquerading as a Level 3 language, rather than a Level 4 language masquerading as a Level 5 (like Starlark, Dhall, even JSONNET). Because of that its syntax is very limited and it needs awkwardly nuanced semantics, and becomes difficult to use well as a result.

HCL is best used when the problem you're solving is nearly one you could use a level 3 language for, whereas in my experience, Starlark is only really worth it when what you need is nearly Python.

The choice between 4 and 5 is more about what you get to avoid. By choosing level 5 you are opening the possibility to make some really complicated configurations and many more footguns. When you stay at level 4 you're forced into using more "standardised" blocks of code that can easily be looked up online and understood.

Level 4 is also far more declarative by nature, you cannot fully compute stuff so a lot is abstracted away declaratively. This also leads to simpler code since you're less encouraged to get into the weeds of instantiation and rather just declare what you'd like.

Overall it's about forcing simplicity by not allowing the scope of possibilities to explode. Certainly there are cases where you can't represent problems cleanly, but I think that tradeoff is worth it because of lowered complexity.

Another benefit of level 4 is that it's easier for your code can stay the same while changing the underlying system you're configuring. Since there's a driver layer between the level 4 configuration and the system which can (ideally) be swapped out.

I don't really like either format (I am not sure which is worse; both have significant problems). YAML has some problems (such as Norway problem and many other problems with the syntax), and JSON has different problems; and some problems are shared between both of them. Unicode is one problem that both of them have. Numbers are a problem in some implementations of JSON but it is not required. (Many other formats have some of these problems too, such as using Unicode, and using floating numbers and not integers, etc.)

I think DER is better (for structured data), although it is a binary format, but it is in canonical form. I made up the TER format which is a text format which you can compile to DER, and some additional types which can be used (such as a key/value list type). While Unicode is supported, there are other (sometimes better) character sets which you can also use.

(However, not all configuration files need structured data, and sometimes programs are also useful to include, and these and other considerations are also relevant for other formats, so not everything should use the same file formats anyways.)

I love that there is one comment saying JSON is better, and then yours saying YAML is better.

anchors/aliases/overrides are one of my favourite yaml features. i've done so much configuration de-duplication with them, it's unreal.

It's a lot easier to accidentally make a file that takes forever than to accidentally make a file that takes a long but finite amount of time.

Lack of nulls in toml is a headache. No two yaml libraries agree on what a given yaml text means. Although json is bad at numbers, that’s more easily worked around.

> YAML has too many footguns (boolean conversions being the first among them)

Copying my own comment from elsewhere: https://news.ycombinator.com/item?id=43670716.

This has been fixed since 2009 with YAML 1.2. The problem is that everyone uses libyaml (_e.g._ PyYAML _etc._) which is stuck on 1.1 for reasons.

The 1.2 spec just treats all scalar types as opaque strings, along with a configurable mechanism[0] for auto-converting non-quoted scalars if you so please.

As such, I really don't quite grok why upstream libraries haven't moved to YAML 1.2. Would love to hear details from anyone with more info.

[0]:https://yaml.org/spec/1.2.2/#chapter-10-recommended-schemas

Hm, you got me thinking about reversible computing and how it could be applied to configuration.

Debugging a configuration becomes tedious once computation is involved. You think some value should be "foo" but it is "bar". Why is it "bar"? If someone wrote it there, the fix is simply to change. If "bar" is the result of some computation, you have to understand the algorithm and its inputs, which is significantly harder.

Given a "reversible" programming language that might be easier. Such languages are weird though and I don't know much about them. For example: https://en.wikipedia.org/wiki/Janus_(time-reversible_computi...

Dave Cunningham created jsonnet from some conversations I had with him about how Nix's lazy language allows one to make recursive references between parts of one's configuration in a declarative way. No need to order the evaluation beforehand.

Dave also designed a way of doing "object oriented" programming in Nix which eventually turned into what is now known as overlays.

P.S. I'm pretty sure jsonnet is Turing complete. Once you get any level of programming, it's very hard not to be Turing complete.

I made a decision early on in a project to replace YAML with jsonnet for configuration and it was the best decision I made on that project - I’ve written tens of thousands of lines of jsonnet since.

The article actually (accidentally or on purpose?) refers to just that:

> How to avoid this madness? Introduce another low-level configuration file. Back to level one...

Every time I work with XML data, I hate it. Just use JSONC imo.