Structured Output with LangChain and Llamafile
14 comments
·June 22, 2025pcwelder
```
try:
answer = chain.invoke(question)
# print(answer) # raw JSON output
display_answer(answer)
print(f"An error occurred: {e}")
chain_no_parser = prompt | llm
raw_output = chain_no_parser.invoke(question)
print(f"Raw output:\n\n{raw_output}")
Wait, are you calling LLM again if parsing fails just to get what LLM has sent to you already?
The whole thing is not difficult to do if you directly call API without Lang chain, it'd also help you avoid such inefficiency.
moribunda
I don't get the langchain hate, but I agree that this "blog post" is bad.
Langchain has a way to return raw output, aside "with structured output": https://python.langchain.com/docs/how_to/structured_output/#...
It's pretty common to use a cheaper model to fix these errors to match the schema if it fails with a tool call.
reedlaw
The use case in the article is relatively simple. For more complex structures, BAML (https://www.boundaryml.com/) is a better option.
dcreater
People still use langchain?
anshumankmr
Its good for quickly developing something but for production, I do not think so.We used it for a RAG application I built last year with a client, ended up removing it piece by piece, and found our app responded faster.
But orgs think its some sort of flagbearer of LLMs.As I am interviewing for other roles now, HRs from other companies still ask for how many years of exp I have with Langchain and Agentic AI.
codestank
i do because i don't know any better since i'm new to the AI space.
nilamo
My experience, as someone who is also new and trying to figure things out, is that langchain works great as long as everything you want to do has an adapter. Try to step off the path, and things get really complex really fast. After hitting that several times, I've found it's easier to just do things directly instead of trying to figure out the langchain way of doing things.
I've found dspy to work closer to how I think, which has made working with pipelines so much easier for me.
zingababba
What should be used instead?
Hugsun
I gave up after it didn't let me see the prompt that went into the LLM, without using their proprietary service. I'd recommend just using the API directly. They're very simple. There might be some simpler wrapper library if you want all the providers and can't be bothered to implement the support for each. Vercel's ai-sdk seems decent for JS.
ebonnafoux
httpx to make the call yourself, or if you really want a wrapper the openAI python https://github.com/openai/openai-python.
null
kristjansson
It's right there. In the screenshot in the blog post. Grammar > 'JSON Schema + Convert'. That's what structured output is.
... it's going to be september forever, isn't it?
The version of llama.cpp that Llamafile uses supports structured outputs. Don't waste your time with bloat like langchain.
Think about why langchain has dozens of adapters that are all targeting services that describe themselves as OAI compatible, Llamafile included.
I'd bet you could point some of them at Llamafile and get structured outputs.
Note that they can be made 100% reliable when done properly. They're not done properly in this article.