OctoAI via VerticalAPI

Supported models

OctoAI models routed by VerticalAPI

Pass the model ID below as model in any OpenAI-compatible request. New OctoAI models are typically supported within 24h of release.

Model ID	Name	Context	Pricing (provider)
`meta-llama-3.3-70b-instruct`	Llama 3.3 70B (Octo)	128K	$0.90 per 1M tok
`qwen2.5-32b-instruct`	Qwen 2.5 32B (Octo)	32K	$0.50 per 1M tok
`stable-diffusion-xl`	Stable Diffusion XL	image	$0.005 per image

Pricing reflects OctoAI's rates — you pay OctoAI directly. VerticalAPI adds zero markup on tokens.

Quickstart

5-line OctoAI call via VerticalAPI

Drop-in replacement for the OpenAI SDK. Works with the OpenAI Python client, Node, Go, curl — anything that speaks HTTP.

                octoai_quickstart.py
                Python
            
from openai import OpenAI

client = OpenAI(
    base_url="https://api.verticalapi.com/v1",
    api_key="vapi_...",
    default_headers={"X-Provider-Key": "..."}
)

response = client.chat.completions.create(
    model="meta-llama-3.3-70b-instruct",  # OctoAI
    messages=[{"role": "user", "content": "Hello"}]
)
print(response.choices[0].message.content)

OctoAI and VerticalAPI

What VerticalAPI does and does not do with OctoAI

Not routed today

VerticalAPI does not proxy OctoAI. This page documents OctoAI's own models and pricing so you can compare it; call OctoAI directly to use it.

What it does route

11 providers behind one OpenAI-compatible endpoint: Anthropic, Cerebras, DeepInfra, Fireworks, Google AI Studio, Groq, Mistral, OpenAI, OpenRouter, Together, xAI. Switchable per request, same SDK.

No token markup on those

Bring your own key and pay the provider directly. VerticalAPI's revenue is the gateway subscription, not a share of your tokens.

Prompts are never stored

Each request is recorded as metadata only — vertical, model, token counts, latency, status. There is no column for prompt or completion text.

Best for

Where OctoAI shines

custom-model hosting image generation (SDXL) OctoStack on-prem deploy fine-tuned variants

Frequently asked questions

What is OctoAI and what models do they offer?

OctoAI (now part of NVIDIA) is an enterprise inference platform. The 2026 catalog includes Llama 3.3 70B, Llama 3.1 8B and 405B, Mixtral 8x7B and 8x22B, Qwen 2.5, plus image models (Stable Diffusion 3, FLUX.1, SDXL) and audio (Whisper). OctoAI's differentiator is custom model deployment — you can bring your own fine-tuned weights and serve them on dedicated NVIDIA GPU endpoints.

How much does OctoAI cost in 2026?

Llama 3.3 70B is roughly $0.90 per 1M tokens (input and output). Llama 405B is in the $3 range. Llama 8B is around $0.15/$0.15. Image generation pricing is per step + resolution. Dedicated endpoints are priced per GPU-hour (typically A100 or H100 classes). Enterprise contracts include volume discounts. You pay OctoAI directly, on OctoAI's own account.

Can I use OctoAI through VerticalAPI?

Create a key at octo.ai, paste it into VerticalAPI, then point the OpenAI SDK at https://api.verticalapi.com/v1. OctoAI exposes OpenAI-compatible endpoints; VerticalAPI passes through, adds unified logging and can fall back to Together, Fireworks or DeepInfra. Custom dedicated endpoints can also be routed by deployment name. Billing stays on your OctoAI account.

What is OctoAI best for compared to alternatives?

OctoAI wins for enterprises that need custom model deployment, fine-tuning ops or NVIDIA-native serving stacks (since the NVIDIA acquisition). Compared to Together or Fireworks it is more turnkey for private deployments and BYO-weights. Compared to AWS Bedrock or Vertex AI it is narrower in catalog but more flexible on custom serving. Not the cheapest path for vanilla open-weight inference.

Where is OctoAI hosted / data privacy?

OctoAI runs on NVIDIA GPU clusters across US datacenters with expansion under NVIDIA. Enterprise tier offers zero data retention, SOC 2 Type II and HIPAA. Dedicated endpoints provide tenant isolation. Your contract and data terms stay between you and OctoAI.

Limitations and trade-offs

Post-NVIDIA-acquisition roadmap and pricing remain in flux — some legacy features are being consolidated into NVIDIA NIM.
Public-tier pricing is competitive but rarely the absolute cheapest vs DeepInfra.
Geographic coverage is US-focused with limited EU/APAC options.
Catalog is narrower than Together for community/niche open-weight models.
Dedicated endpoint setup has more overhead than purely on-demand APIs.

Where OctoAI is heading

Deeper integration with NVIDIA NIM as the merged product matures through 2026.
Expanded dedicated and BYO-weights enterprise tier.
More frequent model catalog refreshes aligned with NVIDIA's optimization stack.
Likely rebranding or consolidation under NVIDIA AI Enterprise.

All supported LLM providers

Same endpoint, same SDK — just change the model and the BYOK header.

Ship on OctoAI in 60 seconds

Free tier — bring your own OctoAI key, zero markup, OpenAI-compatible endpoint.

Get your VerticalAPI key →