Show HN: CodeTracer – A time-traveling debugger implemented in Nim and Rust
58 comments
·March 6, 2025JoeAltmaier
Neat!
Years and years ago I had the opportunity to give Intel processor designers (the time of the 386!) requests for features.
I requested a system tick timer for stamping logs (they did that), bus mask and value registers that triggered a debug interrupt on a match (they did that).
And a jump source history. Maybe 10 jumps back. So on a breakpoint you could figure out how you got there. A time travelling debug feature.
At this point Intel sold an expensive debug probe for recording the bus, you plugged this insane cable into the processor socket and it actually executed in their external hardware, recording every instruction.
My jmp history would have replaced much of that, obviating it's need for the vast majority of users.
Ah well, it didn't happen. So now we all rebuild code 'debug' so we can add tracing and tracking, disrupting the execution path, changing timing and code size and on and on.
I always regretted not getting that.
__init
Intel x86 cores have had Last Branch Records (LBRs) and Branch Trace Store (BTS) since at least Merom in 2006 [1][2]. Nowadays, there's Processor Trace (PT) or Precise Event-Based Sampling (PEBS) which can provide even more information. PT in particular is almost purpose-built to enable this kind of trace reconstruction.
[1] https://stackoverflow.com/questions/14670586/what-is-the-ove...
[2] The MSRs for LBRs (MSR_LASTBRANCH_*_{TO,FROM}_IP) and BTS (IA32_DS_AREA) are described in Volume 4, Section 2.2 of the SDM: "MSRS IN THE INTEL® CORE™ 2 PROCESSOR FAMILY". Core 2 was launched in 2006.
Veserv
Those are only sufficient to produce a execution trace rather than a full data trace which is needed for time-travel debugging.
Though they are sufficient to do what the person you responded to asked for which is just execution trace.
phire
I'm not entirely surprised, adding 40 bytes of SRAM to the 386 for a debug-only feature would have been hard to justify.
cxie
The Noir support makes sense given its use in ZK proofs where execution tracing is particularly valuable, but I'm really looking forward to the Python and Ruby implementations. Those languages' dynamic nature makes bugs particularly elusive sometimes.
Has anyone here tried using this with Noir yet? I'm curious about the performance overhead of the tracing mechanism, especially for longer-running programs. Also wondering if there are plans to support JavaScript/TypeScript for web development use cases.
cxie
The planned RR recordings integration is what I'm most excited about though. Having this capability for systems languages like Rust and C++ would be transformative for complex debugging scenarios where you're often forced to restart debugging sessions from scratch after stepping past a crucial point.
alehander42
the support for system languages (the rr integration "backend") is currently closed source.
It's not ready yet, and it might be proprietary: it would be great if we can open source it, if we find a sustainable business model for that
Apofis
This happens all the time and is super irksome. Being able to step backwards as well as forwards is super cool. Also, being able to do that with a loop using a slider is cool.
cxie
I need a VSCode extension for this. But alas, it's just sitting in their roadmap... Typical. Guess I'll have to roll up my sleeves and build one myself. Not like I have enough on my plate already. At least their trace files are in an open format, so it shouldn't be impossible to hook into the VS Code debugging API.
slifin
If you are using Clojure or ClojureScript check out FlowStorm:
alehander42
Very impressive! Lisp people are always good at tooling.
dloss
Noir is a Domain Specific Language for SNARK proving systems. https://noir-lang.org/
01HNNWZ0MV43FF
I see, I see. And what is a SNARK proving system?
michaelsbradley
This paper is a great resource if you're unfamiliar with zk-SNARK and how it works:
Why and How zk-SNARK Works (2019)
conradludgate
"Succinct Non-interactive Arguments of Knowledge", it's a system for zero-knowledge proofs, which allow proving a fact of some kind without disclosing the inputs
null
throw-the-towel
Just out of curiosity, why did you use two languages to write CodeTracer and not just one of them?
alehander42
Nim is the original language we use. Zahary is a prolific contributor to Nim, and we have a good relationship with the Nim community, they've helped a lot!
Nim and some Python are used for our closed source rr backend currently, and the frontend is written in Nim (compiling to JavaScript).
The backend for blockchain and scripting language which is open sourced, is newer and we used Rust there for several reasons. One of them is related to the fact that many blockchain languages are implemented in Rust and this makes it easier to interoperate/contribute. There are other aspects as well: both languages have pros and cons.
Some pros of Nim are e.g. it's metaprogramming support; the ability to share easily code/types between backend and frontend(it's an alternative to both e.g. C++/Go and TypeScript for us).
We're thankful to both language communities!
rubenvanwyk
Looks really cool, but in production systems, won't the trace files proliferate at extreme speed? How would you correlate the files to a certain session for user identification for example?
alehander42
We are also planning to develop a distributed tracing platform, similar to Jaeger and OpenTelemetry, that continuously records the execution of many distributed processes (e.g. micro-services).
Unlike the existing platforms, which capture only message flows and require you to make educated guesses when some anomaly is observed, our system will let you accurately replay the processing code for each message to quickly identify the root cause for the anomaly.
This would rely on our ability to jump to the specific moment in time when a certain incoming message starts being processed. This moment can be identified either by a log line with a specific format or by a call to some special tracking function (e.g. track_incoming_message(request_id)).
For the system languages, the RR[1] recordings try to be practical by capturing only the non-deterministic events in the program execution. You can pair this with a ring buffer that discards the data after a certain retention period.
For the scripting languages(or any implementation using the db-like traces) we might add some advanced record filtering options.
(But maybe we are misunderstanding the question?)
Veserv
You can not just discard the oldest data of a long-running execution trace when doing replay-based time-travel debugging.
You can not replay execution without a known state followed by all non-determinism after that state which is most easily done by starting from the initial state. To discard data, you need to manifest a state snapshot corresponding to that time to enable forward reconstruction from that state.
alehander42
you're right: in the RR case: currently this is not merged yet, but a RR contributor works on persistent checkpoints; they can act as snapshots
kreco
Especially since the trace files are in .json. [0]
[0] https://github.com/metacraft-labs/runtime_tracing#format
alehander42
True! The next major version of the format should use a more optimized format, as mentioned.
However, some of the important optimizations, that we're preparing are not related so much to the format, but to record more specific things and reconstruct more in the postprocessing.
pzo
I love it, I always wished for something like that. Will try to later test with python. Wish there was also for JS/TS. As for rr debugger did it got by now any support for MacOS or Windows or Android? I'm also wondering how heavy are those recording for typical apps.
This would be also great for LLM to give some context via MCP server or even let LLM pick what variables history wanna see instead of giving full recording file.
Also nice would be some recording filtering that you wanna e.g. pick few variables and display history during whole execution maybe with some specific formating and maybe even for some numeric variables or like tensors, images, etc pass to rerun for visual debugging so you can see plot it
alehander42
Thank you!
The Python initial prototype is not yet finished. It's easy to play with, so anyone interested can actually work on it! Currently, in the experimental tracers, Ruby is usable for smaller programs, so one can try Ruby immediately.
I do plan on improving some of the prototypes, and on adding additional ones: for Lua, but JavaScript: e.g. v8 is also a good target. Scripting language users that find it useful, are welcome to discuss/chat with us, or even directly contribute or propose support for new languages.
A form of record filtering is planned indeed.
We have experimented with automatic chart visualizations of some things, we've planned custom visual representation as well, great to see interest in those
anougaret
I just released a JS/TS/Python time travel debugger that overlays variable values on top of vscode. It's just a npm or pip install and VsCode extension so it might be easier to use for you: https://github.com/dedale-dev/ariana I'm also planning to add MCP integration today actually!
vlovich123
> Wish there was also for JS/TS.
There’s Replay for browsers and Wallaby for Node.
forrestthewoods
Windows support? What languages? How does it work?
I don’t like that the headline is “designed to support multiple languages” but it only actually supports an obscure language I’ve never heard of. Feels like a bait and switch.
alehander42
We're working on Windows support for the scripting and blockchain languages.
I am sorry if the headline felt misleading or the current support disappointing: we do have experimental Ruby support, that you can try right now if you record a `<somepath>.rb` program.
We do design the frontend, trace format/lib and backends to support multiple languages. Ruby is already having experimental support, and we try to keep various other languages/usecases in mind. We hope to find a model that lets us work more on supporting many more scripting languages. We'd also love contributors/the community adding support for languages or codetracer itself!
We also do have a closed source backend based on RR[1] that has partial C/Rust/Nim support, but it is not yet ready. It might be released as a proprietary solution. (However if we find an alternative sustainable business model, it would be great to be able to open source it.)
The scripting/blockchain languages backend is more db-like: it collects a trace by hooking in tracing API-s or instrumenting/patching vm-s (the trace is later postprocessed before replay).
The system languages backend is based on RR[1] recordings currently.
We'd be happy to discuss more usecases or languages!
esafak
Thank you for building up the nim ecosystem.
hugs
came here to say the same. the big problem with nim is not enough people use it. and the way to fix it is with more people using it. (classic catch-22). i struggle with this myself, i might be hiring soon and i know it's going to be hard to find nim programmers. current plan is to recruit python and js developers who wouldn't mind also coding in nim when we'd otherwise need to drop down to the c/c++ layer to integrate with some low-level library for speed/efficiency.
elcritch
Very excited for this! I donated on open collective already. The team is full of talented people. A nice interface to time travel debugging, with Nim support soon nonetheless.
Though if it uses rr it won’t be able to run on macOS. Bummer, macOS seems to get harder and debug on. Luckily lima vms make it easy to remote :/
tester756
Hi alehander42
I've been searching for something like this, so my question is
I have almost identical program in version 1 and 1.01 and I need to find how their behaviour changed
So, I run both of them ./binary1.exe input.txt ./binary2 input.txt and record their execution with your tool
And now, I'd want to extract such data from your tool:
Visited functions and how locals were changing. e.g
int test(int n) {
n++;
std::cout << n << std::endl
n += 15;
if (n > 22)
{
n--;
}
return n + 1;
int main(int argc, char* argv) {
auto result = test(argv[1]);
std::cout << result << std::endl
Visited Function: main with arguments (argc: 2, argv ["path", 7])
Visited Function test with arguments (n: 7)
test: N set to 8
test: N set to 23
test: Entered If (n > 22):
N set to 22
Exit If (n > 22)
test: N set to 23
Exit function test
Main: result set to 23
Exit function main
Can I achieve it with your tool / recording data format?
muizelaar
How does the implementation compare to RR?
alehander42
We are building our future support for the system languages for now directly on top of RR recordings: credit to Robert(roca) and Kyle and all other contributors for RR and Pernosco, they're amazing technologies.
We've researched possible alternative approaches/tools as well, especially keeping in mind Windows/Mac support.
The traces for Noir and the scripting languages work in a completely different way, capturing all the relevant data which is later indexed into a db-like structure. With some future optimizations this can be very useful for various shorter programs in scripting languages, and generally for blockchain languages(as the running time there is usually low) and we hope that eventually with flexible record filtering it can be practical even for capturing important segments/aspects of long-running real world projects.
Hey!
We are presenting CodeTracer - a user-friendly time-traveling debugger designed to support a wide range of programming languages:
https://github.com/metacraft-labs/codetracer?tab=readme-ov-f...
CodeTracer records the execution of a program into a sharable self-contained trace file. You can load the produced trace files in a GUI environment that allows you to move forward and backward through the execution and to examine the history of all memory locations. They say a picture is worth a thousand words — well, a video is even better! Watch the demo below to see CodeTracer in action:
https://www.youtube.com/watch?v=xZsJ55JVqmU
The initial release is limited to the Noir programming language, but CodeTracer uses an open format for its trace files and we've started community-driven projects which aim to add support for Ruby and Python.
We are also developing an alternative back-end, capable of working with RR recordings, which will make CodeTracer suitable for debugging large-scale programs in a variety of system programming languages such as C/C++, Rust, Nim, D, Zig, Go, Fortran and FreePascal.