Groq Function Calling and Tool Use

Every other class of Groq API problem is about a request: it failed, it was slow, it cost too much, it came back in a shape you could not parse. Tool calling is about a decision. You hand the model a set of functions and it chooses whether to act, which function, with which arguments, and when it is finished. Nothing in that sentence is covered by a status code, which is why the failure modes in this guide almost all return HTTP 200 with a perfectly valid body attached to a completely wrong outcome.

A note on scope, because these two get conflated: this guide is not about structured output. Structured output constrains the shape of an answer you already know the model will produce. Tool calling hands over control flow. They compose — most real agents want validated arguments going in and a validated answer coming out — but a JSON schema on the response body does nothing to stop an agent from calling your refund endpoint twice.

The Wire Shape on Groq

Groq uses OpenAI-compatible: tools in a tools array, calls back in choices[0].message.tool_calls, results returned as role "tool" messages keyed by tool_call_id. The mechanical rule that breaks more multi-turn agents than any other: the assistant message containing the tool calls must go back into the conversation verbatim, and every call in it must be answered by exactly one tool-role message carrying the original id. Summarise that assistant turn, drop one of several parallel calls, or regenerate the ids while rebuilding history from your own database, and the next request fails — after your tool already ran and its side effect already happened.

Store transcripts as the provider's own message objects, not as your prettier internal representation. Every team that invents a cleaner schema for agent history rediscovers this on the second turn.

The Description Is the Prompt

A tool that is never called and a tool that is called at the wrong moment are the same bug, and it is nearly always in the description rather than in the model. The model cannot see your implementation. It sees a name, a sentence, and a parameter list, and from those it decides. Most teams write descriptions for the human reading the code and then wonder why the model chose badly.

Three rules carry most of the improvement. Name the trigger conditions in the user's vocabulary, not yours — the description should contain the words a customer would actually type. State the negative case explicitly, because tools get confused with their neighbours far more often than they get invented from nothing. And replace every free-text parameter that has a closed set of valid values with an enum, which converts a whole class of silent wrong-argument bugs into an impossibility.

// Bad: the model has to guess, and it will guess plausibly and wrongly.
{
  name: 'search',
  description: 'Search',
  parameters: { type: 'object', properties: { q: { type: 'string' } } }
}

// Good: the description is the prompt. Enums remove whole failure classes.
{
  name: 'search_orders',
  description:
    'Find orders belonging to the CURRENT customer. Use when the user asks about ' +
    'an order, a delivery, or a refund. Do NOT use for product questions -- use ' +
    'search_catalog for those. Returns at most 20 orders, newest first.',
  parameters: {
    type: 'object',
    properties: {
      status: {
        type: 'string',
        enum: ['pending', 'shipped', 'delivered', 'cancelled'],
        description: 'Omit to search all statuses.'
      },
      since: {
        type: 'string',
        format: 'date',
        description: 'ISO date, YYYY-MM-DD. Omit for no lower bound.'
      }
    },
    required: [],
    additionalProperties: false
  }
}

Keep the set small. Beyond roughly a dozen tools in one call, selection accuracy degrades on every model, and the fix is not a better description — it is a router turn that picks a small tool group first, then a second turn with only that group registered.

The Loop, With Every Guard It Needs

The loop is where agents actually fail, and the naive version — call, execute, call again until the model stops — has no ceiling, no argument validation, no repeat detection, and turns any tool exception into a dead conversation. Each guard below exists because its absence is a known production incident.

// The loop is the product. Everything below is a guard you will wish you had.
const MAX_STEPS = 6;              // hard ceiling: a bug must terminate
const messages = [systemMsg, userMsg];

for (let step = 0; step < MAX_STEPS; step++) {
  const res = await client.chat.completions.create({
    model: PINNED_MODEL,          // tool support is per-model; pin it
    messages,
    tools: TOOLS,
    tool_choice: step === 0 ? 'auto' : 'auto',
  });

  const msg = res.choices[0].message;
  messages.push(msg);             // push the assistant turn VERBATIM

  const calls = msg.tool_calls ?? [];
  if (calls.length === 0) return msg.content;   // model answered: done

  for (const call of calls) {
    const tool = REGISTRY[call.function.name];

    // 1. Unknown tool name -> tell the model, do not throw.
    if (!tool) {
      messages.push(toolMsg(call.id, { error: 'unknown_tool', name: call.function.name }));
      continue;
    }

    // 2. Arguments are model output, i.e. untrusted input. Validate BEFORE executing.
    let args;
    try {
      args = tool.schema.parse(JSON.parse(call.function.arguments));
    } catch (e) {
      messages.push(toolMsg(call.id, { error: 'invalid_arguments', detail: String(e) }));
      continue;               // the model gets to correct itself; the tool never ran
    }

    // 3. Repeat detection: identical call twice in a row is a loop, not progress.
    const fingerprint = call.function.name + ':' + call.function.arguments;
    if (seen.has(fingerprint)) {
      messages.push(toolMsg(call.id, { error: 'repeated_call', hint: 'You already called this with these arguments. Use the previous result.' }));
      continue;
    }
    seen.add(fingerprint);

    // 4. Side effects get an idempotency key derived from the call id.
    try {
      const result = await tool.run(args, { idempotencyKey: call.id });
      messages.push(toolMsg(call.id, result));
    } catch (err) {
      // 5. Tool failures are DATA, not exceptions. The model can route around them.
      messages.push(toolMsg(call.id, { error: 'tool_failed', message: err.message }));
    }
  }
}

// 6. Ceiling hit. This is an incident signal, not a silent fallback.
metrics.increment('agent.step_ceiling_hit', { model: PINNED_MODEL });
throw new AgentStepLimitError(MAX_STEPS, messages);

// Every tool result must be a tool-role message carrying the ORIGINAL call id.
// Dropping or rewriting that id is the single most common multi-turn break.
function toolMsg(id, payload) {
  return { role: 'tool', tool_call_id: id, content: JSON.stringify(payload) };
}

Two of those deserve emphasis. Returning tool failures as data rather than exceptions is what separates an agent that degrades gracefully from one that dies mid-task: a model told that the lookup failed will apologise or try another route, exactly as a competent human would. And the step ceiling should be an alert, not a silent fallback — a task that legitimately needs more steps than you budgeted is a design change, while a task that loops is a defect, and you want to know which one you are looking at.

Forcing, Preventing, and Parallel Calls

tool_choice is the cheapest correctness lever in the whole surface and it is routinely left on auto for turns where the answer was never in doubt. An extraction turn should force a specific named tool: prose is not an acceptable output there, so do not allow it and then write retry logic to catch it. A final summarisation turn should set none, so a model that has already gathered everything cannot decide on one more lookup. Auto is for genuinely open turns, which is fewer of them than most agents assume.

Parallel calls are a real latency win for independent lookups and a real hazard for a handler written on the assumption of one call per turn. The failure is quiet: you execute the first element of the array, ignore the rest, and the next request fails or the model proceeds believing it received answers it never got. Iterate over every call, emit one tool message per id, and for anything with a side effect prefer serialisation plus an idempotency key over concurrency.

Arguments Are Untrusted Input

A tool argument is a value written by a language model that read whatever your user typed. That makes it exactly as trustworthy as a form field, and the fact that it arrived via an API response rather than a POST body changes nothing. Parse it against a strict schema, reject unknown properties instead of quietly ignoring them, and enforce authorisation in your own code — scope every query to the current session's tenant rather than to an identifier the model supplied.

The dangerous shape is a tool that accepts a record id and looks it up without a tenant check, because a persuasive user message can convince a model to ask for someone else's record and your API will return it with a clean 200. This is where tool calling meets prompt injection: the injected instruction does not need to break your parser, it only needs to persuade the model to call a tool you were willing to expose.

Tool-Calling Failure Modes, and How They Present

Note the status-code column. Five of these six return 200, which is why uptime dashboards and error-rate alerts see nothing while the agent misbehaves.

The Groq Trap: Tool Support Is a Property of the Model, Not of the Endpoint

Groq serves open-weight models behind an OpenAI-compatible surface, and that compatibility is about the wire format, not about capability. The endpoint will accept a tools array for any model you name; whether the model behind it was trained to emit tool calls is a separate question with a per-model answer. Send tools to a model that does not support them and you do not get a 400 telling you so -- you get a normal completion in which the model describes, in prose, the function it would like you to call. Your parser looks at tool_calls, finds it empty, and reports that the model refused.

This makes model selection part of your agent's correctness, not just its cost. Pin the model, confirm on that exact model that a trivial forced call comes back in the tool_calls array rather than in content, and re-run that check whenever you change the model string. A one-request assertion in CI is enough, and it fails loudly on the day someone swaps the model for a cheaper one that cannot do this.

The second Groq-specific trap is economic rather than behavioural. Groq's whole proposition is speed, and a runaway tool loop on a fast provider does not announce itself the way it does elsewhere. On a slow provider, an agent stuck calling the same function repeatedly is obvious within seconds because the request hangs. At Groq's token rates the same loop completes fifteen iterations before a human notices anything, returns something plausible, and bills you for all of it. The step ceiling below is not a formality here -- it is the only thing between a schema bug and a token bill.

What to Log

Agent observability is not request observability. Log, per turn: the number of steps the task consumed, every tool name chosen with the outcome of its argument validation, the count of repeated-call rejections, and whether the step ceiling was hit. Those four numbers turn "the agent feels flaky" into a specific broken tool description. Step-count distribution in particular is the single best health metric an agent has — when its tail starts creeping toward your ceiling, something has stopped working before anyone complains.

What your own logs cannot tell you is whether Groq itself is the reason a step failed, because they only contain your traffic. That is the gap external monitoring fills.

Frequently Asked Questions

Related Guides

• Groq Structured Output and JSON Mode
• Groq API Error Codes Explained
• Groq Prompt Injection Defence
• Groq Context Window Limits
• Groq Model Deprecation and Version Drift
• How to Test and Mock the Groq API
• Groq API Rate Limits
• Live Groq Status Check