Be Aware of the Makefile Effect

With that attitude how would the presently accepted languages/frameworks have come about?

> "designing a whole new system around one pet peeve"

BAHHAHAH! So…you mean React. If I hear the word hook as if it alone can solve complexity in web dev one more time I’ll…eh, I’ll do nothing actually. But my point still stands. React solves asynchronous event driven behavior well, but that’s all. Everything else in React projects is, well, everything else.

The 10x watchmaker, perhaps?

You can "favorite" comments

If you spend 80% of your time (and mental energy) applying the knowledge you already have and 20% learning new things, you will very quickly be able to win more battles per day than someone who spends 1% of their time learning new things.

Specifically for the examples at hand:

- at 20%, you will be able to write a Makefile from scratch within the first day of picking up the manual, rather than two or three weeks if you only invest 1%.

- if you don't know what the CPU registers are, the debugger won't be able to tell you why your C++ program dumped core, which will typically enable you to resolve the ticket in a few minutes (because most segfaults are stupid problems that are easy to fix when you see what the problem is, though the memorable ones are much hairier.) Without knowing how to use the disassembly in the debugger, you're often stuck debugging by printf or even binary search, incrementally tweaking the program until it stops crashing, incurring a dog-slow C++ build after every tweak. As often as not, a fix thus empirically derived will merely conceal the symptom of the bug, so you end up fixing it two or three times, taking several hours each time.

Sometimes the source-level debugger works well enough that you can just print out C++-level variable values, but often it doesn't, especially in release builds. And for performance regression tickets, reading disassembly is even more valuable.

(In C#, managed C++, or Python, the story is of course different. Until the Python interpreter is segfaulting.)

How long does it take to learn enough assembly to use the debugger effectively on C and C++ programs? Tens of hours, I think, not hundreds. At 20% you get there after a few dozen day-long debugging sessions, maybe a month or two. At 1% you may take years.

What's disturbing is how many programmers never get there. What's wrong with them? I don't understand it.

> I completely encourage anyone to learn as much about the tools and stack as possible, but there is only so much time.

That seems like a weird way to think about this. I mean, sure, there's no time today to learn make to complete your C++ ticket or whatever. But yesterday? Last month? Last job?

Basically, I think this matches the upthread contention perfectly. If you're a working C++ programmer who's failed to learn the Normal Stable of Related Tools (make, bash, python, yada yada) across a ~decade of education and experience, you probably never will. You're in that 50% of developers who can't start stuff from scratch. It's not a problem of time, but of curiosity.

This is the 40% that OP mentioned. But there's a proportion on people/engineers that are just clueless and are incapable of understanding code. I don't know the proportion so can't comment on the 50% number, but hey definitely exist.

If you never worked with them, you should count yourself lucky.

We can’t really call the field engineering if this is the standard. A fundamental understanding of what one’s code actually makes the machine do is necessary to write quality code regardless of how high up the abstraction stack it is

Funny enough I'm an A̶I̶ML researcher and started in HPC (High Performance Computing).

  >  if I’m a C++ programmer ... should I know how the CPU registers are called?

Especially with "low level"[0] languages knowing some basics about CPU operations goes a long way. You can definitely get away without knowing these things but this knowledge will reap rewards. This is true for a lot of system based information too. You should definitely know about things like SIMD, MIMD, etc because if you're writing anything in C/C++ these days it should be because you care a lot about performance. There's a lot of stuff that should be parallelized that isn't. Even stuff that could be trivially parallelized with OpenMP.

  > what should an AI researcher working always in Jupyter know?

I think it is easy to not recognize that information is helpful until after you have that information. So it becomes easy to put off as not important. You are right that it is really difficult to balance everything though but I'm not convinced this is the problem with those in that category of programmers. There's quite a number of people who insist that they "don't need to" learn things or insist certain knowledge isn't useful based on their "success."

IMO the key point is that you should always be improving. Easier said than done, but it should be the goal. At worst, I think we should push back on anyone insisting that we shouldn't be (I do not think you're suggesting this).

[0] Quotes because depends who you talk to. C++ historically was considered a high level language but then what is Python, Lua, etc?

That's why suggestions like RTFM! are stupid. I just don't have time to read every reference documentation of every tool I use.

If everything in your tree is similar, yes. I agree that's going to be a very small Makefile.

While this is true, for much larger projects, that have lived for a long time, you will have many parts, all with slight differences. For example, over time the language flavour of the day comes and goes. Structure changes in new code. Often different subtrees are there for different platforms or environments.

The Linux kernel is a good, maybe extreme, but clear example. There are hundreds of Makefiles.

Sure, but this will require you to know how to tell the compiler to generate your Makefile header dependencies and if you end up making a mistake, this will cause silent failures.

I'd say: you are absolutely using the right tool. :-)

You're probably using the wrong tool and should consider a simple plain shell script (or a handful of them) for your tasks. test.sh, build.sh, etc.

I’d be curious to hear your ratio. It really varies. In some small teams with talented people, there are hardly any “fake” developers. But in larger companies, they can make up a huge chunk.

Where I am now, it’s easily over 50%, and most of the real developers have already left.

PS: The fakes aren’t always juniors. Sometimes you have junior folks who are actually really good—they just haven’t had time yet to discover what they don’t know. It’s often absolutely clear that certain juniors will be very good just from a small contribution.

pretty sure we've made this complaint about a subset of developers since way before chatgpt and the like.

> You generally can't run them locally

I recognize that this is such a disincentive for me taking the initiative to fiddle with and learn about anything like this

The local part is my big problem too. I used azure Dev ops in work. I find clicking through the UI to be a miserable experience, Id love to have it running locally so I could view inputs and outputs on the file system. Also yaml is an awful choice, no one I know enjoys working with it. The white space issues just get worse and worse longer your files get.

> You generally can't run them locally

GitLab CI gives you local runners. You can completely self-host CI.

+1. The only CI tool that I've seen really organize around this principle is Buildkite, which I've used and enjoyed. I'm currently using Github Actions and it's fine but Buildkite is literally sooooo good for the reasons you've mentioned.

If you think cmake is a good example of more complex than make, then you haven't seen automake/autoconf. The first thing I thought of. You can find tons of tons of configure scripts that check if you're running ancient versions of Unix, checks that a byte is 8 bits wide, and a ton of other pointless checks. They don't do anything with all that information, don't think for a moment that you can actually build the app on Irix, but the checks for it have been passed along for decades likes junk DNA.

I started using ccache to speed up Make, but soon found that allowed me to replace Make entirely with a bash script using a few functions.

I disagree, but I think once a project gets beyond a certain level of complexity you may need to move beyond make. For simple projects though I usually do something like:

    CC=clang
    MODULES=gtk+-3.0 json-glib-1.0
    CFLAGS=-Wall -pedantic --std=gnu17 `pkg-config --cflags $(MODULES)`
    LDLIBS=`pkg-config --libs $(MODULES)`
    HEADERS=*.h
    EXE=app
    
    ALL: $(EXE)

    $(EXE): application.o jsonstuff.o otherstuff.o

    application.o: application.c $(HEADERS)

    jsonstuff.o: jsonstuff.c $(HEADERS)

    otherstuff.o: otherstuff.c $(HEADERS)

    clean:
            rm -f $(EXE) *.o

I use makefiles all the time for my projects; projects that are actually built with something else (ex, gradle, maven, whatever). My makefiles have targets for build, clean, dependencies, and a variety of other things. And they also have inputs (like "NOTEST=true") for altering how they run. And then I use make to actually build the project; so I don't need to remember how the specific build tool for _this_ project (or the one of many build tools in a project) happens to work. It works pretty well.

The idea that git offers a 'clean' command was revelatory to me. Your build system probably shouldn't need to know how to restore your environment to a clean state because your source control should already know what a clean state is.

That's sort essential to serving its purpose, after all.

I haven't yet run into a scenario where there was a clean task that couldn't be accomplished by using flags to git clean, usually -dfx[0]. If someone has an example of something complex enough to require a separate target in the build system, I'm all ears.

[0] git is my Makefile effect program. I do not know it well, and have not invested the time to learn it. This says something about me, got, or both.

That’s why I usually write them from scratch and don’t let them get over 100 lines long at most. Usually they are around 30 with white space.

make clean makes lots of sense but is not even strictly necessary. In the world where all it does is find all the *.o files and deletes them it’s not a bad thing at all.

What if your client comes back?

On a more practical note, what structure, formats and tools do you use that enable you to feed it to an LLM?

I would almost call this the "boilerplate effect".

Where people copy the giant boilerplate projects for React, K8, Terraform, etc. and go from there. Those boilerplates are ideal for mid to large scale projects. And it's likely you'll need them someday. But in the early stages of development it's going to impart a lot of architecture decisions that really aren't necessary.

Seeing this exact effect where I am currently working. Main available CI/CD tool is a customised and centrally managed Jenkins fleet. It's pretty much impossible to avoid using and seldom needs changed - until it does. Some attempts have been made at centralised libraries and patterns - but even that requires knowledge and study that most won't know is available or be given time to acquire.

So when the inevitable tweak or change is made it's made in the easiest, cheapest way - which is usually copying an existing example, which itself was copied from somewhere else.

I see exactly the same in other teams repositories. Easiest path taken to patch what already exists as the cost/benefit just isn't perceived to be there to worth prioritising.

  > only worked within a broader context that is now missing

  > because the thing being copied—the code—actually is effectual in its own right.

In fact, a cargo cult works because there's the appearance of a casual linkage. It appears things work. But as we know with code, just because it compiles and runs doesn't mean "it works". It's not a binary thing. Personal I find that belief is at the root of a lot of cargo cult development. Where many programmers glue things together say "it works" because they passed some test cases but in reality code shouldn't be a Lovecraftian monster made of spaghetti and duct tape. Just because your wooden plane glides doesn't mean it's AC an actual plane

It’s always hoped (but rarely shown to be true) that by making templates, teams will put thought into their K8s deployments etc. instead of just copy/pasting. Alas, no – even when the only things the devs have to do is add resource requests and limits, those are invariably copy/pasted. If the app gets OOMkilled, they bump up memory limit until it doesn’t. If it’s never OOMkilled, it’s probably never touched, even if it’s heavily over-provisioned (though that would only matter for the request, of course).

This has spawned a cottage industry of right-sizing tooling, which does what a dev team could and should have done to begin with: profiling their code to see resource requirements.

At this point, I feel like continuing to make things easier is detrimental. I certainly don’t think devs need to know how to administer K8s, but I do firmly believe one should know how to profile one’s code, and to make reasonable decisions based on that effort.

I agree, and I think the key distinction is in understanding. In a cargo cult there's a lack of understanding, whereas I'll often copy and paste code/config I understand to get something done. Usually this is for something I don't do very often (configuring nginx, writing some slightly complicated shell script etc.) I could spend an hour reading docs and writing the thing from scratch but that's likely gonna be wasted time because there's a good chance Im not going to look at that thing again for a few years.

Pretty much this

And of course every one of those tools has to have their own special language/syntax that makes sense nowhere else (think of all the tools beyond make, like autotools, etc)

I don't care about make. I don't care learning about make beyond what's needed for my job

Sure, it's a great tool, but I literally have 10 other things that deserve more of my attention than having my makefile work as needed

So yeah I'll copy/paste and be done with it

Copy-pasting itself is not bad per se. What's bad is copy-pasting without understanding the why and how.

For LaTeX I also copy-paste a whole lot from older files, but I don't feel bad because (a) I wrote these files before, (b) I know exactly what each line is doing, (c) I understand why each line is needed in the new doc.

I wrote a relatively large amount of TikZ code earlier in my life (basically used it as a substitute for Illustrator) and for this library in particular, I think it just has so much syntax to remember that I cannot keep it all in my brain for ever. So I gladly copy from my old TikZ code.

\usepackage[utf8]{inputenc} now is the default, at least; you don't need to include it anymore. And diacritics work out of the box, no need to write weird incantations like G\"{o}del anymore.

I wouldn't say so. Make is very simple and you can grasp the basis within an hour or so, if you're familiar with shell scripting (as it's basically a superset of shell scripts, with the dependency graph on top). Then all you have to do is just in time learning which is mostly searching for a simpler pattern that what you're currently doing.

Hum...

You know, a makefile is documentation. That's why you should probably never copy one (except for a single line here or there). There's space for commenting a few stuff, but your target names and variables should explain most of what is going there.

Anyway, the article and most people here seem to be talking about those autotools generated files. Or hand-built ones that look the same way. But either way, it's a bad solution caused by forcing a problem to be solved by a tool that wasn't aimed at solving it. We have some older languages without the concept of a "project" that need a lot of hand-holding for compiling, but despite make being intentionally created for that hand-holding, it's clearly not the best tool for that one task.

Exactly. Bonus points if the person who started the project moved on and you have to be the one to build and maintain it.

You find the first part in your stack that is documented (e.g., make is documented, even if your makefile is not) and use that documentation to understand the undocumented part. You then write down your findings for the next person.

If you don’t have enough time, write down whatever pieces you understood, and write down what parts “seem to work, but you don’t understand“ to help make progress towards better documentation.

If you put the documentation as comments into the file, this can make copy&pasting working examples into a reasonably solid process.

I am that someone else because I seldom edit the makefiles and I forget things. That's why I try to trim unused targets and recipes and I try to keep it documented.

In the end it is no different from any code that's suffered from 10 years of tuning and it can get ugly. Maybe Make is even somewhat worse in this respect, but then again it does not need to be changed often.

I would let the expert do his job and I do mine.

Which in my experience sometimes involves copying last years justifications