Stack Traces Are Underrated
39 comments
·March 10, 2025bob1029
pjc50
> A broader outcome of this enlightenment is preference for monolithic products. Stack traces fare poorly across web service and API boundaries. If you've only ever worked with microservice architectures, the notion of a stack trace may seem distracting.
Yes. People forget that the original concept of microservices, the AWS "everything must have an API", was to put in an accountability boundary across teams. Either the API behaves per its contract or it does not, you're neither expected nor really allowed to cross that boundary into the API to find out why it's doing that.
In an environment which is correctly doing "each microservice is a different small team", that helps. In an environment which is doing "one team maintains lots of microservices", this is nearly always an anti-pattern.
the_mitsuhiko
> But Rust has a better workaround to create stack traces: the backtrace module, which allows capturing stack traces that you can then add to the errors you return. The main problem with this approach is that you still have to add the stack trace to each error and also trust library authors to do so.
That's technically true, but the situation is not as dire. Many errors do not need stack traces. That so few carry a backtrace in Rust is mostly a result of the functionality still not being stable [1].
The I think bigger issue is that people largely have given up on stack traces I think, in parts because of async programming. There are more and more programming patterns and libraries where back traces are completely useless. For instance in JavaScript I keep working with dependencies that just come minified or transpiled straight out of npm. In theory node has async stack traces now, but I have yet to see this work through `setTimeout` and friends. It's very common to lose parts of the stack.
Because there are now so many situations where stack traces are unreliable, more and more programmers seemingly do lose trust in them and don't see the value they once provided.
I also see it in parts at Sentry where a shocking number of customers are completely willing to work with just minified stack traces and not set up source maps to make them readable.
badmintonbaseba
Python asyncio supports meaningful stack traces through async functions just fine.
import asyncio
async def baz():
await asyncio.sleep(.1)
raise RuntimeError()
async def bar():
await asyncio.sleep(.1)
await baz()
async def foo():
await asyncio.sleep(.1)
await bar()
async def main():
await asyncio.sleep(.1)
await foo()
if __name__ == "__main__":
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
main_task = loop.create_task(main())
try:
loop.run_until_complete(main_task)
except KeyboardInterrupt:
main_task.cancel()
loop.run_until_complete(asyncio.wait([main_task]))
pass
$ python3 test_stacktrace.py
Traceback (most recent call last):
File "/home/lenard/tmp/test_stacktrace.py", line 24, in <module>
loop.run_until_complete(main_task)
File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
return future.result()
File "/home/lenard/tmp/test_stacktrace.py", line 17, in main
await foo()
File "/home/lenard/tmp/test_stacktrace.py", line 13, in foo
await bar()
File "/home/lenard/tmp/test_stacktrace.py", line 9, in bar
await baz()
File "/home/lenard/tmp/test_stacktrace.py", line 5, in baz
raise RuntimeError()
RuntimeErrorreseasonable
Not sure about node (and I don’t recall it ever being a problem), but chrome supports stack traces through setTimeout just fine.
I’m not sure there are many reputable modules on npm that minify without source maps, and if people aren’t using them I’d consider them to be making a poor technical choice, one that I would correct before contributing to the project.
Diffing two lengthy stack traces to find a divergence is perhaps the fastest way to debug a slew of bug types. Let alone just the ability to instantly click into a file/line even from console prints as you follow the execution path.
And my favorite part is being able to ignore / hide external modules and specific files in chrome’s debugger which allows for stepping through only your code, and evaluating much shorter traces. Something java needed decades ago.
When I do use print debugging I always use console.error to include the expandable stack trace as needed, I can’t imagine how slow it would be to not have that always, and have to resort to stepping and breakpoints to get around.
the_mitsuhiko
> I’m not sure there are many reputable modules on npm that minify without source maps, and if people aren’t using them I’d consider them to be making a poor technical choice, one that I would correct before contributing to the project.
React is a good example of a library that is a transpiled mess when installed from npm. Sadly not the only one, there are many more popular libraries that look like this.
vlovich123
I wish there was a mode to force Errors to automatically capture traces & print them as part of the chain on panic. Would save a lot of time when debugging & let you force libraries into supporting it.
johncolanduoni
One nice side effect of how Rust’s Futures work is that in many cases “normal” stack traces actually reflect the async/await flow accurately. You should see a series of “poll” methods called on each future in the async call chain.
the_mitsuhiko
Only until you spawn it into an executor :(
CMDBob
A stack trace (or even better, a minidump with the call stack!) is one of the most useful debugging things for me. Hell, the call stack in general is super useful to me!
I can look at a stack trace, go "oh, function X is misbehaving after being called by function Y, from function Z", and work out what's gone wrong from the context clues, and other debugger info. As a game developer, with codebases that are big, semi-monolithic codebases, it's essential, especially when code crosses the gameplay/engine and engine/kernel barriers.
windward
>Are they just not used to having them so that they don't miss them?
The languages that I work in that don't print useful traces are typically strongishly-typed system languages. So I miss them - sometimes having to step through offending lines of code in a debugger - but I also completely avoid a whole class of bugs that are responsible for most of my stack traces in Python.
TFA's example isn't one of these, but is a function that would have a return code checked and logged if erroneous. This class of bug also can't be inlined and makes an easy breakpoint-ee.
montebicyclelo
Great point. I've found it's very often possible to understand and fix problems "one shot" from stack traces alone — and we're talking production builds here... So I wouldn't turn them off, (an idea mentioned in the article), unless profiling shows that they are one of the last things preventing the code from reaching the target performance.
albertzeyer
Stack traces are very valuable. Sometimes it can even help to attach them to some object creation, when you later wonder why/how/where this object was created. E.g. in TensorFlow, every single Tensor had a traceback attached to it, so when there was any error later on, it would show you where it was created. This is maybe less needed now with eager mode, but you might have other similar situations.
One problem with stack traces is maybe that they can be too verbose. E.g. if you print them for any warning you print to log (or stdout). Sometimes they will be extremely helpful for debugging some problem, but in many cases, you maybe don't need them (you know why you get the warning and/or you don't care about it).
You could also add more information to the stack trace such as local variables. That can be even more helpful for debugging then, but again adds more verbosity.
For example, we often use this to add information about relevant local variables: https://github.com/albertz/py_better_exchook
One solution to the problem with verbosity is when you have foldable text output. Then the stack trace is folded away (not shown in all details) and you can unfold it to see the details. See the DomTerm demo here: https://github.com/albertz/py_better_exchook#domterm
Some more on text folding:
https://github.com/PerBothner/DomTerm/issues/54
https://gitlab.com/gnachman/iterm2/-/issues/4950
https://github.com/xtermjs/xterm.js/issues/1875
https://gitlab.freedesktop.org/terminal-wg/specifications/-/...
creshal
Stack traces are underrated, unless you're developing EnterpriseJavaSingletonFactoryAbstractionFactoryFactories, in which case they're buffer overflows on your poor log analyzer
zokier
Kinda related, but I feel it would be useful for log entries to include file/lineno and/or some unique identifier. Helps both pinpointing where some weird message comes from, and for searching for specific entries in the logs.
Sure, you can grep the log message but it can be difficult if it has some templating/formatting going on, and it can be pretty easy to end up with non-unique messages.
XorNot
Whats weird is how expensive this can be - i.e. to do it in Go requires invoking runtime reflection, whereas technically the compiler should be able to update the final numbers into the messages at build time.
cm2187
In C# the quasi mandatory async/await for everything has many downsides, particularly for debugging. It breaks all stack traces. It also makes it impossible to pause the code.
HdS84
Ha? Can you give an example? I've seen lots of perfectly good stack traces in async code - no problems at all. Pausing code also works, at least using vs or rider.
TinkersW
They are useful sure, and I print a stacktrace on any type of error/exception, but often breaking into the debugger is even more useful and faster as you can see local variables, program state, and what other threads happen to be doing.
piva00
Hard to break into the debugger for a production application running on hundreds of servers.
Cthulhu_
One can argue whether stack traces should be enabled for production (at least on all servers) given they're relatively expensive to create. Which isn't a problem if they're exceptional, but in a lot of cases they aren't.
TinkersW
Perhaps, but remote debugging is a thing, though triggering an auto break into debugger would be more complex.
piva00
Remote debugging is a thing but not very practical when you have hundreds of instances of your application running across dozens/hundreds of servers.
For smaller scale apps it's a godsend but I haven't worked in that environment in more than a decade so remote debugging is essentially useless in my work.
anonzzzies
I am a big fan of Lisp SBCL stack traces; even in complex projects I never saw before, I'm almost always able to read, interpret and fix the issue just from that.
Chilinot
I have been an avid proponent of the way errors are managed in Rust and Go for a long time. However, this article raises a very good point. Before i started developing in Rust and Go, i did Java and python for several years. And damn, do i miss those stacktraces every now and then when something bad happens that isn't properly handled by the code.
Still, i do think returning the error as a return value is better than having a completely separate flow when dealing with exceptions. I like that it forces me to properly deal with an error and not just ignore it and think something like "meh, i'll get to this later". Because i will never "get to it later".
karl42
You could combine both by adding a stack frame each time the error is returned one level up. This could be done explicitly (cumbersome and not everyone will do it) or automatically by the language (weird magic, but useful).
Stack traces are your #1 ally when supporting someone else's legacy production pile.
Once you get comfortable with how they work and what information they contain, you can hit the ground running anywhere. Stack traces will teach you about the product architecture faster than anyone on the team can.
As you embrace them, you take the little bit of extra time to make sure they go well. For example, re-throwing exceptions correctly, properly awaiting results, etc. Very minor details that make all the difference.
A broader outcome of this enlightenment is preference for monolithic products. Stack traces fare poorly across web service and API boundaries. If you've only ever worked with microservice architectures, the notion of a stack trace may seem distracting.