Program-of-Thought Prompting Outperforms Chain-of-Thought by 15% (2022)

Anthropic recently added this to the API: https://www.anthropic.com/engineering/advanced-tool-use

See "Programmatic Tool Calling"

And there was an AI productivity startup called Lutra AI doing this, although they've since pivoted to some kind of MCP infra thing: https://lutra.ai/

This seems to be incorporated into current LLM generations already -- when code execution is enabled both GPT-5.x and Claude 4.x automatically seem to execute Python code to help with reasoning steps.

Underlying paper is from 2022 and should be indicated in the title.

I call that self-destructive prompting in the sense that you use AI to output programs that replace calling the AI in the future. The paper seems to indicate that this also brings much better results. However it's subject to attacks as running generated code is usually unsafe. A sandbox has to be used, major agentic AI players are providing some solutions, like Langchain sandbox released earlier this year.