Infinite Tool Use
5 comments
·May 25, 2025anko
dijit
I was once told that we can only hold 7 things in our heads at once, especially smart people might manage 9; this was by a psychologist that I respect- whether its true or not I am not certain. He was using it as an argument to either condense the array of things I was thinking about into smaller decisions, or to make decisions and move on instead of letting them rot my brain.
It was good advice for me.
blixt
Let’s not forget that every round trip with the LLM costs latency (and extra input tokens). We now have parallel tool calls which sometimes works in some models[1]. But it’s great because now a model can say “write these 3 files then read these 2 files” before the time-to-first token latency is incurred once more (not to mention input token cost).
I think LLMs will indirectly move towards being fuzzy VMs that output tokens much like VM instructions so they can prepare multiple conditional branches of tool calling, load/unload useful subprograms, etc. It might not be expressed exactly like that, but I think given how LLMs today are very poor at reusing things in their context window, we will naturally add features that take us in this direction. Also see frameworks like CodeAct[2] etc.
[1] This can be converted to a single tool call with many arguments instead, which you’ll see providers do in their internal tools, but it’s just messier.
fny
This is a paradigm I’ve been using well before all this tool nonsense.
You have a file which defines a series of typed functions. You ask the LLM to generate code for a task given those functions and then send* that code to an executor which runs the code.
If you need tool calls you make those inside of functions which call an own LLM with that capability.
You can do a lot of crazy things this way, plus you can very easily audit what’s happening. You also can have very specific contexts tied to nested LLM calls.
brador
Your only useful purpose is to assign the goal. Everything else is an uppity human getting in the way of a more efficient (and more creative) production system.
I have been thinking along these lines myself. Most of the time, if we need to calculate things, we'd use a calculator or some code. We wouldn't do it in our head, unless it's rough or small enough. But that's what we ask LLMs to do!
I believe we juggle 7 (plus or minus 2) things in our short term memory. Maybe short term memory could be a tool!
We also don't have the knowledge of the entire internet in our heads, but meanwhile we can still be more effective at strategy/reasoning/planning. Maybe a much smaller model could be used if the only thing it had to do is use tools and have a basic grasp on a language.