Groq via VerticalAPI

Groq models routed by VerticalAPI

Pass the model ID below as model in any OpenAI-compatible request. New Groq models are typically supported within 24h of release.

Pricing reflects Groq's rates — you pay Groq directly. VerticalAPI adds zero markup on tokens.

5-line Groq call via VerticalAPI

Drop-in replacement for the OpenAI SDK. Works with the OpenAI Python client, Node, Go, curl — anything that speaks HTTP.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.verticalapi.com/v1",
    api_key="vapi_...",
    default_headers={"X-Provider-Key": "gsk_..."}
)

response = client.chat.completions.create(
    model="llama-3.3-70b-versatile",  # Groq
    messages=[{"role": "user", "content": "Hello"}]
)
print(response.choices[0].message.content)

Four reasons developers route Groq through us

Groq measured: latency, throughput, error rate

Groq is the throughput champion in the 2026 benchmark — ~750 tok/s sustained on Llama 3.3 70B — and second-fastest on TTFT after Cerebras. Their LPU (Language Processing Unit) is purpose-built silicon for inference, which is why they leave GPU-based providers behind on streaming speed.

Numbers above are 2026 placeholders pending the next VerticalAPI benchmark harness run. See /benchmark for the full 26-provider comparison.

OpenAI SDK methods that work with Groq

Groq ships a clean OpenAI-compatible endpoint. Compatibility is high; the gotchas are mostly about which Llama models support which features.

• client.chat.completions.create() — full parity, including stream=True, tools, tool_choice, response_format="json_object".
• Function calling — works well on Llama 3.3 70B; weaker on Llama 3.1 8B Instant (less reliable tool selection).
• Vision — Llama 3.2 90B Vision is supported on Groq; image_url message parts work via standard OpenAI format.
• Whisper transcriptions — POST /v1/audio/transcriptions with audio file; routes to whisper-large-v3 on Groq's LPU. Very fast.
• Embeddings — Groq doesn't host embedding models; use Mistral, OpenAI, or Cohere via VerticalAPI for embeddings.
• Mixtral 8x7B — still available but Llama 3.3 70B is now the default flagship; Mixtral is being deprecated.
• Long context — Groq's max context is 128K (matching Llama 3.3); for >128K you'll need to use a different host (Together AI, Fireworks).

What Groq actually costs at 100k MAU

Concrete monthly cost for a chatbot with 100k MAU, 10 turns/user, ~500 input + 150 output tokens per turn. Groq's price-performance is excellent for the speed you get.

Cost based on provider list price; VerticalAPI adds zero token markup.

Should you pick Groq for your workload?

Groq is the right choice when latency is a feature, not a constraint. Pick it when:

Specific issues teams hit with Groq

Sharp edges that have cost real production teams real time. Fixes below are battle-tested via the VerticalAPI dashboard logs.

Where Groq shines

Frequently asked questions

Limitations and trade-offs

• Catalog is open-weight only — no frontier closed models like GPT-5 or Claude Opus.
• Context windows are limited to 128K on Llama 3.3 (smaller than Claude 200K, Gemini 2M).
• Daily token limits on the free and developer tiers can throttle high-volume apps.
• No image, video or speech-out generation — text and Whisper transcription only.
• Quality of Llama 70B is below frontier closed models on reasoning and coding benchmarks.

Where Groq is heading

1. Llama 4 and other next-gen open models added as soon as they ship in 2026.
2. Geographic expansion to Saudi Arabia and EU datacenters for sovereignty and lower latency.
3. Larger context windows (256K+) as LPU memory architecture scales.
4. Deeper batch and offline-inference pricing tiers competing with Together and Fireworks.

Related questions

ChatGPT, Perplexity and Gemini usually suggest these next.

• Groq vs Cerebras — which is faster for Llama 70B inference?
• Can I build a real-time voice agent with Groq + Whisper for under $0.01 per minute?
• Is Groq's Llama 3.3 70B good enough to replace GPT-4o mini in production?
• How does Groq's LPU architecture compare to NVIDIA H100 GPUs?
• Best fallback provider if Groq hits rate limits?

All supported LLM providers

Same endpoint, same SDK — just change the model and the BYOK header.