Gemma3 Function Calling

So in short there's no guarantee for any output from any LLM whether its Gemma or any other (ignoring some details like setting a random seed or parameters like temperature to 0). Like you mentioned though libraries like outlines can constrain the output, whereas hosted models often already include this in their API, but they can do so because its a model + some server side code.

With Gemma, or any open model, you can use the open libraries in conjunction to get what you want. Some inference frameworks like Ollama include structured output as part of their functionality.

But you mentioned all of this already in your question so I feel like I'm missing something. Let me know!

But I think you already mentioned all this in your response so I might be missing the question?

If you run Gemma via Ollama (as recommended in the Gemma docs) you get exactly that feature, because Ollama provides that for any model that they run for you: https://ollama.com/blog/structured-outputs

Under the hood, it is using the llama.cpp grammars mechanism that restricts allowed logits at each step, similar to Outlines.

We feel this model excels at instructability which is why we're recommending bringing your own prompt! Benchmark wise you can see this performance from BFCL directly, they (independently) ran their eval using their prompted format the larger Gemma models performed quite well if you ask me.

Specifically though I want to thank you for leaving a comment. We're reading all this feedback and its informing what we can do next to reduce frustration and create the best model experience the community

That's insightful. Thank you for sharing your work and the patient responses to everyone's questions.

Yesterday I started exploring a smaller Gemma3 model locally with Ollama, and it's clearly a level up from the previous model I was using (Llama3) in terms of instruction comprehension and the sophistication of responses. It's faster, smaller, and smarter.

I very much appreciate how such innovative technology is available for non-experts to benefit from and participate in. I think one of the best things about the emergence and evolution of LLMs is the power of open source, open standards, and the ideal of democratizing artificial intelligence and access to it. The age-old dream of machines augmenting the human intellect (Vannevar Bush, Doug Englebart, et al) is being realized in a surprising way, and seeing the foundational layers being developed in real time is wonderful.