Google’s FunctionGemma is one of the most interesting small models released recently — a 270 million parameter model that does exactly one job: turning natural language into structured function calls. It’s small enough to run on a laptop, a Raspberry Pi, or even a smartphone. But there’s an important catch that trips up many first-time users: Google explicitly designed this model to be fine-tuned. So what can you actually do with it as-is, and when does fine-tuning become mandatory? Let’s dig in.
What is FunctionGemma?
FunctionGemma is a specialized version of Google’s Gemma 3 270M model, trained specifically for function calling — translating requests like “turn on the flashlight” or “create a calendar event for lunch” into executable API calls. The key specs:
- 270M parameters — tiny by modern standards (most “small” models are 1B+)
- 32K token context window — enough room for large tool schemas and multi-turn history
- Text-only — no image or audio input
- Extremely low memory footprint — the full-precision model runs in roughly 550 MB of RAM on CPU, meaning almost any device can host it
- Not a chat model — Google is clear that FunctionGemma is not intended for general dialogue. It’s a router, not a conversationalist.
The design philosophy is worth understanding: instead of one giant model doing everything, FunctionGemma acts as a fast, private, local “translator” between human language and your software’s defined actions. Google demonstrated this with two showcase apps: Tiny Garden, a voice-controlled game where the model decomposes commands like “plant sunflowers in the top row” into game functions, and Mobile Actions, an agent that triggers Android system tools — all running fully offline on a phone.
What works without fine-tuning
The base model (functiongemma-270m-it) is genuinely usable out of the box for a specific class of tasks. Zero-shot, it can:
1. Handle simple, single-turn tool calls with clear schemas. If you pass a small set of well-named functions with obvious parameters (get_weather(city), set_timer(minutes)), the base model can map straightforward requests to the right call. Community fine-tunes aside, many developers use the stock model for prototyping exactly this way.
2. Decide when NOT to call a function. The model card reports a BFCL Irrelevance score of 70.6 in zero-shot mode — meaning it’s reasonably good at recognizing when a user request doesn’t match any available tool, instead of hallucinating a call.
3. Serve as a prototyping base. Because it uses a standard chat-template workflow (you pass your tool schemas via the tokenizer’s chat template), you can wire it into an existing pipeline in an afternoon and see how far the base model gets before investing in training.
Practical zero-shot tips:
- Use the exact system prompt the model was trained with: “You are a model that can do function calling with the following functions” — deviating from it degrades output noticeably.
- Keep temperature very low (around 0.1). You want deterministic routing, not creativity.
- Keep the tool count small and the parameter names self-explanatory. The fewer and clearer your functions, the better zero-shot performance you’ll get.
- Always validate the output against your schema in code. At this model size, malformed or wrong calls will happen — your application layer should catch them, not your users.
When fine-tuning becomes necessary
Here’s the honest picture from Google’s own benchmarks: on the Mobile Actions task, the base model achieved 58% accuracy — and after fine-tuning on the task-specific dataset, that jumped to 85%. That gap is the whole story of small models. A 270M model doesn’t have the general reasoning capacity to handle ambiguity, so specialization is where it earns its keep.
You should plan to fine-tune when:
- Your function schema is domain-specific. Custom app actions, niche APIs, or unusual parameter structures that don’t resemble generic examples.
- You need multi-turn behavior. Conversations where the model must track state across turns and chain calls.
- You need production-grade consistency. Google’s positioning is explicit: FunctionGemma is for teams that want the deterministic behavior of a specialized model rather than the variability of zero-shot prompting.
- You’re working in a non-English language or mixed-language input. Community projects (for example, a Vietnamese banking fine-tune) show this works well — but it requires training data in your language.
How fine-tuning actually works (it’s easier than you think)
The good news: fine-tuning a 270M model is about as accessible as fine-tuning gets in 2026.
The standard recipe is Unsloth + LoRA on a free Google Colab GPU. Unsloth provides ready-made starter notebooks (built in collaboration with Google) that load the model, apply optimized LoRA fine-tuning, and handle the chat template correctly. Because the model is so small, training runs that would take hours for larger models finish in minutes, and VRAM requirements are minimal.
The workflow looks like this:
- Build a dataset of examples: available tools + user prompt + the correct function call output. Google’s Mobile Actions dataset on Hugging Face is the reference format to copy. You don’t need millions of rows — for a narrow schema, hundreds to a few thousand well-crafted examples go a long way. A common trick is using a frontier model to generate the synthetic training pairs.
- Train with the Unsloth notebook (Colab or local — note that local Unsloth training isn’t supported on Apple Silicon yet).
- Export to GGUF (Q8_0, F16, or BF16) directly from the notebook.
- Run locally in LM Studio, Ollama, or llama.cpp — or go further and deploy to a phone. Google and PyTorch published a quantization-aware training workflow that gets roughly 50 tokens/second on devices like the Pixel 8 and iPhone 15 Pro.
The decision framework
Ask yourself three questions:
- Is my tool set small, generic, and single-turn? → Try the base model first. It may be good enough, especially with strict output validation.
- Do I need high accuracy on a custom schema, multi-turn flows, or a specific language? → Fine-tune. The 58% → 85% jump on Mobile Actions shows what specialization buys you.
- Do I need general conversation plus tool use? → FunctionGemma is the wrong tool entirely. Use a larger general model, or pair FunctionGemma (as the router) with something else for dialogue.
Final thoughts
FunctionGemma represents a genuinely different way of thinking about AI deployment: instead of renting intelligence from a cloud API, you train a tiny specialist that runs on hardware you already own — private, offline, and essentially free at inference time. Out of the box it’s a capable prototype engine; fine-tuned, it becomes a production-grade router for your specific application. For developers building on modest hardware, that combination is hard to beat.
For more hands-on tech and code guides, see our Tech & Code hub.
Useful resources: the FunctionGemma model card on Hugging Face (google/functiongemma-270m-it), Google’s Mobile Actions fine-tuning guide at ai.google.dev, and Unsloth’s FunctionGemma documentation. Model specs and benchmark figures change as new versions ship — verify against the current model card before you build.