Lessons from interviews on deploying AI Agents in production
59 comments
·November 4, 2025reedf1
What you actually need in most business cases is a 100% auditable, explainable and deterministic workflow. While AI is strictly deterministic - it is technically chaotic. Introducing this in large customer pipelines or in intensive data applications means that even if the AI only does something a bit off 99%, 99.9% or 99.99% you will see large spurious error rates in your workflow. Worst of all these will be difficult to explain - or maybe even purposely hidden, as I have seen some agents attempt to do.
IanCal
You absolutely don’t need this. We know this to be true as we use humans and they are none of these things (at 100%) and we use other ml systems that don’t hit all there. Directionally those things are beneficial but you just need the benefits to outweigh the costs.
bandrami
Yeah no. I make software used on actual flight simulators and we literally need it to be deterministic, to the extent of needing the same help query to always return the exact same results for all users at all times.
IanCal
Some business problems need that. That’s not the same as asserting most do and it’s certainly not the same all business problems.
Some things need to be deterministic. Many don’t.
Even your business will have many such problems that don’t need 100% all those properties - every task performed by a human for example. You as a developer are not all of these things 100%!
And your help query may need to be deterministic but does it need to be explainable? Many ml solutions aren’t really explainable, certainly not to 100% whatever that may mean, but can easily be deterministic.
charcircuit
If you were on a real flight and asked a human for help, they wouldn't give a deterministic answer. This doesn't seem like an actual requirement that is needed, but rather something that is post hoc rationalized because it was cheaper to make that way. While terms like consistency may come up when referring to having deterministic output as a requirement, the true reason could actually just be cost.
kakacik
Depends on the industry, clearly you never worked in such. Regulated (medical, transport, municipality, state, army and so on) or just with some decent enforced regulations like whole finance for example, and bam! you have serious regulatory issues that every single sane business tries desperately to stay away from.
null
IanCal
“There are business problems” and “most business problems” are not the same thing.
flir
How can the agent hide the error?
You log the interaction, you see what happened, no?
wongarsu
In coding agents that would be "the test keeps failing and I can't fix it - let's delete the test" or "I can't fix this bug, let's delete the feature"
If you measure success by unit test failures or by the presence of the bug those behaviors can obscure that the LLM wasn't able to do the intended fix. Of course a closer inspection will still reveal what happened, but using proxy measurements to track success is dangerous, especially if the LLM knows about them or if the task description implies improving that metric "a unit test is failing, fix that"
flir
Sure, but the discussion here is around "in production"? I'm trying to imagine a scenario and I'm coming up short.
Joel_Mckay
The people outside the business of selling hype will not be keen on paying to break their business with popular liabilities. =3
enraged_camel
This comment is a bit strange.
>> While AI is strictly deterministic - it is technically chaotic
AI is neither deterministic nor chaotic. It is nondeterministic because it works based on probability, which means that for open-ended contexts it can be unpredictable. But properly engineered agentic AI workflows can drastically reduce and even completely eliminate the unpredictability. Having proper guardrails such as well-defined prompts, validations and fallbacks in place can help ensure mistakes made by AIs don't result in errors in your system.
throw-qqqqq
> AI is neither deterministic nor chaotic. It is nondeterministic because it works based on probability
A deterministic function/algorithm always gives the same output given the same input.
LLMs are deterministic if you control all parameters, including the “temperature” and random “seed”. Same input (and params) -> same output.
viccis
Ignoring that you are making an assumption about how the randomness is handled, this is a very vacuous definition of "deterministic" in the context of the discussion here, which is AI controlling large and complex systems. The fact that each inference can be repeated if and only if you know and control the seed and it is implemented with a simple PRNG is much less important to the conversation than its high level behavior, which is nondeterministic in this application.
If your system is only deterministic if it processes its huge web of interconnected agentic prompts in exactly the same order, then its behavior is not deterministic in any sense that could ever be important in the context of predictable and repeatable system behavior. If I ask you whether it will handle the same task the same exact way, and its handling of it involves lots of concurrent calls that are never guaranteed to be ordered the same way, then you can't answer "yes".
mejutoco
I thought this too, but it seems that is not the case. I could not remember the reason I saw why so I googled it (AI excerpt).
Large Language Models (LLMs) are not perfectly deterministic even with temperature set to zero , due to factors like dynamic batching, floating-point variations, and internal model implementation details. While temperature zero makes the model choose the most probable token at each step, which is a greedy, "deterministic" strategy, these other technical factors introduce subtle, non-deterministic variations in the output
District5524
Not that it's incorrect but there is some data showing variability even with the very same input and all parameters. Especially if we have no control over the model behind the API with engineering optimizations etc. See Berk Atil et al.: Non-Determinism of "Deterministic" LLM Settings, https://arxiv.org/abs/2408.04667v5
cnnlives78
The LLMs most of us are using have some element of randomness to every token selected, which is non-deterministic. You can try to attempt to corral that, but statistically, with enough iteration, it may provide nonsense, unintentional, dangerous, opposite solutions/answers/action, even if you have system instructions defining otherwise and a series of LLMs checking themselves. Be sure that you fully understand this. Even if you could make it fully deterministic, it would be deterministic based on the model and state, and you’ll surely be updating those. It amazes me how little people know about what they’re using.
jampekka
I wouldn't be surprised if autoregressive LLMs had some chaotic attractors if you stretch the concept to finite discrete state (tokens).
CjHuber
Are they? I mean I wouldn't say they are strictly deterministic, but with a temperature and topk of 0 and topp of 1 you can at least get them to be deterministic if I'm correct. In my experience if you need a higher temp than 0 in a prompt that is supposed to be within a pipeline, you need to optimize your prompt rather than introduce non determinism. Still of course that doesn't mean some inputs won't give unexpected outputs.
Incipient
Can you share some examples of eliminating non-determinism? I feel like I should be able to integrate agents into various business systems, but this issue is a blocker.
Eg. An auto email parser that extracts an "action" - I just don't trust that the action will be accurate and precise enough to execute without rereading the email (hence defeating the purpose of the agent)
_joel
I'm not sure it eliminates but reducing the temperature and top-k/p?
belter
> Introducing this in large customer pipelines or in intensive data applications means that even if the AI only does something a bit off 99%, 99.9% or 99.99% you will see large spurious error rates in your workflow.
You just described how you get your google account locked... :-)
advikipedia
We recently spoke with 30+ startup founders and 40+ enterprise practitioners who are building and deploying agentic AI systems across industries like financial services, healthcare, cybersecurity, and developer tooling.
A few patterns emerged that might be relevant to anyone working on applied AI or automation:
- The main blockers aren’t technical. Most founders pointed to workflow integration, employee trust, and data privacy as the toughest challenges — not model performance.
- Incremental deployment beats ambition. Successful teams focus on narrow, verifiable use cases that deliver measurable ROI and build user trust before scaling autonomy.
- Enterprise adoption is uneven. Many companies have “some agents” in production, but most use them with strong human oversight. The fully autonomous cases remain rare.
- Pricing is unresolved. Hybrid models dominate; pure outcome-based pricing is uncommon due to attribution and monitoring challenges.
Infrastructure is mostly homegrown. Over half of surveyed startups build their own agentic stacks, citing limited flexibility in existing frameworks.
The article also includes detailed case studies, commentary on autonomy vs. accuracy trade-offs, and what’s next for ambient and proactive agents.
If you’re building in this space, the full report is free here: https://mmc.vc/research/state-of-agentic-ai-founders-edition...
Would be interested to hear how others on HN are thinking about real-world deployment challenges — especially around trust, evaluation, and scaling agentic systems.
Etheryte
Perhaps I simply don't understand what you mean, but it sounds like the first point could be rephrased in some way. To me, workflow integration and data privacy sound very much like technical blockers.
advikipedia
More than the "actual" problem, the "perception" of the problem is worse. Workflow integration is more to do with users having to rethink their workflows, their roles, and how they work with AI. As for data privacy concerns, even where startups have taken measures to overcome the problems, very often enterprises still remain concerned (making this more of a perception problem than an actual problem). That's why I focused on the non-technical aspect of it!
DrScientist
When I see vendors complain about workflow and integration issues, it's because the vendors software is written around an expectation of a certain workflow and integration points and they find out in reality every customer does it slightly differently.
Some key challenges around workflow are that while the fundamental white-board task flow is the same, different companies may distribute those tasks between people and over time in different ways.
Workflow is about flowing the task and associated information between people - not just doing the tasks.
Same goes for integration - the timing of when certain necessary information might be available again not uniform and timing concerns are often missed on the high level whiteboard.
Here's a classic example of ignoring timing issues.
https://www.harrowell.org.uk/blog/2017/03/19/universal-credi...
barrenko
But if you define them as non-technical related blockers agents are just swell.
refactor_master
Consider this simple example: Storing all your sensitive user data in one centralized location (e.g. a US server) would be great for any kind of analytics and modeling to tap into, and is technically very easy to do, but it also violates virtually every country's data privacy laws. So then you have to set up siloed servers around the world, deal with data governance, legal stuff, etc.
Sure, it then becomes a technical challenge to work around those limits, but that may be cost/time prohibitive.
1718627440
That sounds more like, that you can solve the problem, when it would have other requirements.
IanCal
There are two sides to workflow integration.
One is technical (it’s a hassle to connect things to a specific system because you’d need to deal with the api or there is no api)
The other isn’t, because it’s figuring out how and where to use these new tools in an existing workflow. Maybe you could design something from scratch but you have lots of business processes right now, how do you smoothly modify that? Where does it make sense?
Frankly understanding what the systems can and can’t do takes at least some time even if only because the field is moving so fast (I worked with a small local firm who I was able to help by showing them the dramatic improvements in transcription quality vs cost recently - people here are more used to whisper and the like but it’s not as common knowledge how and where you can use these things).
baxtr
> The main blockers aren’t technical. Most founders pointed to workflow integration, employee trust, and data privacy as the toughest challenges — not model performance.
What does that even mean? Are you trying to say that the problem isn’t that the AI models are bad — it’s that it’s hard to get people to use them naturally in their daily work?
Arnechos
For example where I work business users required model output to be 100% correct, which wasn't possible, so they decided to stick to old manual workflow.
sigwinch
That’s our definition of a process: when your objective is well-defined, a process is guaranteed to succeed. Not everything is a process. And sometimes people mistake what the desired success must be. For example, a piece of surgical equipment might not have features guaranteeing profitability.
thatjoeoverthr
Honestly, just sad seeing AI posts on HN now.
ChrisMarshallNY
I’m not sure this is the case, here (although it’s always a possibility, sadly).
It just looks like the highly-polished marketing copy I’ve read, all my career. It’s entirely possible that it was edited by AI (a task that I have found useful), but I think that it’s actually a fairly important (to the firm) paper, and was likely originally written by their staff, and carefully edited.
I do feel as if it’s a promotional effort, but HN often features promotional material, if it is of interest to our community.
donatj
I've seen companies including my own pouring lots of money into AI. Outside of "replacing developers", I am genuinely curious what have people done that's actually useful?
We've got a sort of "business intelligence" AI they poured a lot of time and money into, and I don't think anyone really uses it because it makes stuff up.
I'm sure there are things. I just haven't seen them. I would love to hear concrete examples.
The cynic in me says I wouldn't want something with the error aptitude and truth telling of a small child taking any sort of important action on my behalf.
sigwinch
We had a lack of (digital images of) training cases in emergency surgery. You’d prefer to give ML experience with many rare cases, but must resort to style transfer. Humans can do this, but variously problematic and you’re taking them away from necessary work.
roxolotl
> The main blockers aren’t technical. Most founders pointed to workflow integration, employee trust, and data privacy as the toughest challenges — not model performance.
These, outside of employee resistance are technical problems. The insistence they aren’t seems to be the root of the misunderstanding around these tools. The reality is that “computers that speak English” are, at face value, incredibly impressive. But there’s nothing inherent to said systems that makes them easier to integrate with than computers which speak C. In fact I’d argue it’s harder because natural languages are significantly less precise.
Communication and integration is incredibly challenging because you’re trying to transfer states between systems. When you let “the machine carry a larger share of the burden,” as Dijkstra described of the presumed benefit of natural language programming but actual downside[0], you’re also forfeiting a large amount of control. It is for the same reason that word problems are considered more challenging than equations in math class. With natural languages the states being communicated are much less clear than with formal languages and much of the burden assumed to be transferred to the machine is returned in the form of an increase in required specificity and preciseness of which formal languages already solve for.
None of this is to say these tools aren’t useful nor that they cannot be deployed successfully. It is instead to say that the seduction of computers which speak English is more exactly that. These tools are incredibly easy to use to impress, and much more challenging to use to extract value.
0: https://www.cs.utexas.edu/~EWD/transcriptions/EWD06xx/EWD667...
IanCal
I think I disagree here.
The integration parts aren’t natural language issues but connecting systems and how to put these things in your workflow.
For example. I have a bunch of off the cuff questions and problems and tasks. I want to have these in one place and have that trigger a conversation with ChatGPT, which shows the results in the first place but can be continued easily.
Before it was added the other week, I could track issues in linear and I could have codex write the code to solve them but only by manually moving text from one place to another, manually moving the tickets, checking on progress, click g buttons to open prs - all of that is integration hassle but none is about the model itself. I think now with GitHub copilot I could just assign a task.
roxolotl
The point is because natural language systems seem so easy to use people assume the existing hard parts will become easy. But integrating systems is hard not because formal languages are challenging to use but because connecting systems is inherently challenging. The switch from formal to natural language doesn’t reduce challenge it just alters it.
We’re saying the same thing. Integration is the hard (still technical) part.
sanskarix
most teams skip the obvious wins - start with the stuff nobody wants to do. invoice extraction, meeting notes, basic support triage. boring problems with clear success metrics.
the fancy autonomous agents can wait. prove value on tasks where 80% accuracy is already better than nothing, then move up the chain.
advikipedia
That's precisely what we found in our research as well! We outlined it in our observations too (excerpt below):
The most successful deployment strategies we’ve seen started with:
simple and specific use cases with clear value drivers, that were low risk yet medium impact;
weren’t majorly disruptive to existing workflows;
preferably automating a task that the human user dislikes (or was outsourced);
the output of the workflow can be easily/quickly verified by the human for accuracy or suitability; and
demonstrated clear ROI quickly
Given the current levels of technological development, AI Agents work best when narrowly applied to very specific tasks and operating under a specific context. For instance, we’ve seen this in healthcare with revenue cycle management processes (claim and denial management) that health systems were already outsourcing to third-party providers.
The land-and-expand strategy for AI agents is very different to traditional SaaS. Given enterprises are increasingly under pressure from the C-Suite to incorporate AI into their work, there are plenty of opportunities for startups to “land” but it’s much harder to “expand” – and not only that, it’s taking much longer to expand even when they want to expand, because it’s a use case by use case rollout.
Much like the iconic Volkswagen ad, sometimes it’s better to “Think Small” and build trust first, rather than attempt too many use cases (and excessively complex use cases) right off the bat.
DrScientist
> Much like the iconic Volkswagen ad, sometimes it’s better to “Think Small” and build trust firsy
Poor choice of example for building trust - Volkswagen: - lie big on emissions/fuel efficiency performance.
arisAlexis
All of these are temporal. It's the only thing we can be sure of. Very limited value in these "lessons" 6 months ahead.
jgoode19
[dead]
OBELISK_ASI
[dead]
veegee
[dead]
jakozaur
I talked to some enterprises and saw similar patterns:
1. Agentic AI systems are hard to measure and evaluate methodologically.
2. Quote from Salesforce analyst day: "it's been so easy to build a killer demo, but why has it been so hard to get agents that actually deliver the goods.”
3. Unfortunately, small errors tend to compound over time, which means most systems need a human in the loop as of 2025.
4. A lot of enterprise buyers feel the huge potential (and FOMO), yet ROI is still unclear as of 2025. MIT report "State of AI in business 2025": Despite $30–40 billion in enterprise investment into GenAI, 95% of organizations are not seeing profit and loss impact.
thatjoeoverthr
The Salesforce analyst quote is ChatGPT, though.
jakozaur
thatjoeoverthr
It may have come out of his mouth, but people don’t say “killer” as an adjective anymore. Not for a long time. It’s one of ChatGPT’s standard anachronisms.
The 'incremental deployment' insight is the quiet winner here. Most teams treat agents like feature launches when they're closer to organizational rewiring. You can't A/B test trust. The gap between 'demo magic' and 'production resilience' isn't about better prompts—it's about designing for graceful degradation and building feedback loops that survive contact with actual users. Homegrown stacks make sense until you realize you're reinventing observability for the third time.