Most people meet ChatGPT first and the OpenAI API second. The gap between them trips up a lot of builders. ChatGPT is a finished product; the API is a set of raw endpoints you wire into your own app, script, or automation. You get the same underlying models, but nobody hands you a chat window, a memory system, or guardrails. You build those.
This guide is for the person who’s decided they need programmatic access and wants to know what they’re actually signing up for, before the first invoice shows up.
Key takeaways
- The API charges per token (input and output), not per message. Long context and chatty outputs are where costs sneak up.
- You don’t need to use the newest model for everything. Cheaper models handle most classification, extraction, and drafting work fine.
- The Chat Completions endpoint covers 90% of use cases. Reach for Assistants, Realtime, or the Responses API only when you have a specific reason.
- Set a hard spending limit on day one. A runaway loop can burn real money fast.
- Rate limits, not price, are what stall most early projects. Plan around them.
What the OpenAI API actually gives you
At its core, the API is an HTTP endpoint you send text (or images, or audio) to, and it sends a completion back. You authenticate with a secret key, pick a model, pass some messages, and read the response. That’s the whole loop.
What’s genuinely useful is the surrounding pieces. Function calling (now usually called tool calling) lets the model return structured JSON that your code can act on, so you can have it decide “the user wants to cancel an order” and hand you a clean object instead of a paragraph. Structured Outputs can force the response to match a schema you define, which kills most of the parsing headaches. There’s also embeddings for search and similarity, image generation, speech-to-text, and text-to-speech, all through the same account.
One thing worth being blunt about: the API has no memory. Every request is stateless. If you want a chatbot that remembers what was said three messages ago, you resend that history every single time. That’s not a bug, but it does mean your token count grows with the conversation, and so does your bill.
How billing works, and where it bites
You pay per token. A token is roughly three-quarters of an English word. Both what you send (input) and what you get back (output) count, and output is usually priced higher. Prices differ by model and change over time, so I won’t quote figures that’ll be stale by next quarter. Check the official pricing page before you commit.
Here’s the part that surprises people. The cost isn’t driven by how many times you call the API. It’s driven by how much text moves each time. A single request that stuffs a 40-page document into context can cost more than a hundred short questions.
Three habits that keep the bill sane:
- Trim the conversation history you resend. Summarize old turns instead of pasting them verbatim.
- Cap output length with a max tokens setting so a model can’t ramble for pages.
- Match the model to the job. Reasoning-heavy models cost more and think longer; a lighter model answers a formatting or extraction task for a fraction of the price.
Set a monthly usage limit in your account settings on the first day. Not later. The classic disaster is a bug that loops an API call thousands of times overnight, and the only thing standing between you and a nasty invoice is that hard cap.
Which model should you pick?
OpenAI keeps several model families active at once, and the naming shifts. Rather than chase specific version numbers, think in tiers based on what a task needs.
| Task type | What to reach for | Why |
|---|---|---|
| Classification, tagging, data extraction | A small/fast model | The work is mechanical; you’re paying for speed and low cost, not deep reasoning. |
| Chatbots, drafting, summaries | A mid-tier general model | Good balance of quality and price for high-volume, everyday text. |
| Multi-step reasoning, code, hard analysis | A reasoning model | It thinks longer and costs more, but gets tricky logic right where cheaper models slip. |
| Voice, images, transcription | The dedicated audio/image models | These are separate endpoints tuned for the modality. |
A practical way to decide: start with the cheapest model you think might work, test it on ten real inputs, and only move up a tier if the quality is genuinely failing. Plenty of teams overpay by defaulting to the flagship model for tasks a lightweight one handles cleanly.
The endpoints, and when each one earns its keep
You’ll see several ways to talk to the models. Choosing the wrong one adds complexity you don’t need.
- Chat Completions is the workhorse. Send messages, get a reply. Use this unless you have a reason not to.
- The Responses API is a newer, more flexible interface that bundles tool use and state handling. Worth exploring for new projects, but Chat Completions is more heavily documented across the community.
- Assistants is a higher-level layer with built-in threads, file handling, and tools. It saves setup for certain apps but hides some control, and it’s had shifting status. Don’t build critical infrastructure on it without checking its current support state.
- Realtime handles low-latency voice conversations. Only relevant if you’re building something that talks back in real time.
- Embeddings turns text into vectors for semantic search and retrieval. This is what powers “answer questions about my documents” features, paired with a vector database.
If you’re building a retrieval system (feeding the model your own docs), that’s the embeddings plus Chat Completions combo, not a magic “upload your PDF” button. The API won’t do the retrieval logic for you unless you use a higher-level tool that wraps it.
Getting your first call working
- Create an account on the OpenAI platform and add a payment method. Free trial credit, when offered, expires, so plan for paid usage.
- Generate an API key under your account settings. Copy it once; you can’t view it again later. If you lose it, you make a new one.
- Store the key in an environment variable, never in your code or a public repo. Leaked keys get abused, and you pay for the abuse.
- Install the official SDK (Python or Node are best supported) or just send a plain HTTP POST request.
- Send a minimal Chat Completions request with a model name and one user message. If you get a reply object back, you’re connected.
- Set your monthly spending limit before you build anything real.
Failure signals to watch at this stage: a 401 means your key is wrong or not loaded; a 429 means you hit a rate limit or ran out of quota; a 400 usually means a malformed request body. Read the error message, it names the field.
Common mistakes that cost money or time
Hardcoding the API key into a script and pushing it to GitHub. Automated bots scan public repos for keys within minutes and start spending. Use environment variables and, if it leaks, revoke it immediately.
Resending the entire chat history unpruned. On a long conversation this quietly multiplies your token cost per message. Summarize or truncate old turns.
Assuming the model remembers previous requests. It doesn’t. If your app “forgets” context, you forgot to send it.
Ignoring rate limits until you launch. New accounts start with low limits that raise over time as you spend. If your launch plan needs high throughput on day one, request a limit increase early and design retries with backoff.
Not handling non-deterministic output. Even with the same prompt, responses vary. If your app depends on a fixed format, use Structured Outputs or validate before you trust the result.
Who this is not for
If you just want to chat with an AI, use ChatGPT. The API adds cost, security work, and code for no benefit unless you’re building something programmatic.
If you need a no-code chatbot on your website, a wrapper tool that sits on top of the API will get you there faster than raw endpoints. You’d only go direct when you need full control over prompts, data handling, or cost.
The API makes sense when you’re automating a workflow, embedding AI into a product, processing data at scale, or building anything that runs without a human clicking a button each time.
FAQ
Is the OpenAI API free?
No, it’s usage-based. New accounts sometimes get trial credit that expires, but ongoing use is paid per token. There’s no free tier that stays free for production traffic.
What’s the difference between ChatGPT Plus and the API?
They’re billed and used completely differently. A ChatGPT Plus subscription is a flat monthly fee for the chat app. The API is pay-as-you-go for programmatic access, and a Plus subscription doesn’t include API credit. They’re separate products on separate billing.
Can the API read my PDFs or files?
Not by itself in a plug-and-play way through the basic endpoint. You either extract the text and feed it in as context, build a retrieval system with embeddings, or use a higher-level tool that handles files for you. The core Chat Completions call just takes text and images you supply.
How do I keep costs under control?
Set a hard monthly limit in your account, cap output length per request, prune the conversation history you resend, and use the smallest model that does the job. Monitor the usage dashboard in your first weeks so surprises stay small.
Do I need to know how to code?
To use the API directly, yes, at least enough to make HTTP requests and handle responses. If you don’t code, look at automation platforms and chatbot builders that connect to the API for you.
Start small. Get one real call working, watch the token counts, then scale up once you understand what each request costs you. The API rewards people who measure before they build big.
Related articles