How to Reduce OpenAI API Costs by 80% Without Sacrificing Quality

OpenAI costs can spiral fast once you go to production. One client showed us their first month's bill with the expression of someone who had opened a credit card statement and found a very expensive typo. It was not a typo. Here are the techniques we use on every project to keep the numbers from going parabolic.

1. Use the right model for the task

This is the biggest lever. Developers default to GPT-4o for everything — it's like hiring a senior surgeon to label your spreadsheets. GPT-4o-mini costs ~17x less and handles most tasks just as well.

MODEL_MAP = {
    "classify": "gpt-4o-mini",
    "summarize": "gpt-4o-mini",
    "extract": "gpt-4o-mini",
    "reason": "gpt-4o",
    "code": "gpt-4o",
}

2. Cache aggressively

Same input → same output. Cache at the prompt level.

import hashlib, json
from functools import wraps

_cache = {}

def cached_llm_call(func):
    @wraps(func)
    async def wrapper(prompt: str, **kwargs):
        key = hashlib.sha256((prompt + json.dumps(kwargs, sort_keys=True)).encode()).hexdigest()
        if key in _cache:
            return _cache[key]
        result = await func(prompt, **kwargs)
        _cache[key] = result
        return result
    return wrapper

@cached_llm_call
async def classify(text: str) -> str:
    # ... OpenAI call
    pass

For production, use Redis instead of an in-memory dict:

import redis
r = redis.Redis()

async def cached_call(prompt: str, ttl: int = 3600) -> str:
    key = f"llm:{hashlib.sha256(prompt.encode()).hexdigest()}"
    if cached := r.get(key):
        return cached.decode()
    result = await call_openai(prompt)
    r.setex(key, ttl, result)
    return result

3. Trim prompts ruthlessly

Every token costs money. Audit your prompts:

import tiktoken

enc = tiktoken.encoding_for_model("gpt-4o")

def count_tokens(text: str) -> int:
    return len(enc.encode(text))

def trim_to_budget(text: str, max_tokens: int) -> str:
    tokens = enc.encode(text)
    if len(tokens) <= max_tokens:
        return text
    return enc.decode(tokens[:max_tokens])

Common token waste:

• Repeating instructions the model already follows
• Including entire documents when only a section is relevant
• Verbose few-shot examples when the model already understands the task

4. Batch requests

Instead of one API call per item, batch multiple items into one prompt:

# ❌ 100 API calls
for email in emails:
    category = await classify_email(email)

# ✅ 1 API call
prompt = f"""Classify each email as: sales / support / spam.
Return JSON array matching the order of emails.

Emails:
{json.dumps([e.body for e in emails], indent=2)}"""

results = await call_openai(prompt)
categories = json.loads(results)

Batching 20 items per call cuts API overhead by ~95%.

5. Use prompt caching (Anthropic) or batch API (OpenAI)

Anthropic's prompt caching charges 10% of normal price for cached prefix tokens. If you have a long system prompt that doesn't change between calls, mark it for caching:

response = anthropic.messages.create(
    model="claude-sonnet-5",
    max_tokens=1024,
    system=[{
        "type": "text",
        "text": long_system_prompt,
        "cache_control": {"type": "ephemeral"}  # cache this
    }],
    messages=messages
)

OpenAI's Batch API processes requests asynchronously at 50% cost. Use it for non-real-time workloads (nightly processing, bulk analysis):

batch = client.batches.create(
    input_file_id=file_id,
    endpoint="/v1/chat/completions",
    completion_window="24h"
)

6. Set max_tokens tight

Default max_tokens is often 4096. If you only need a 50-word classification label, cap it:

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=messages,
    max_tokens=50,  # not 4096
)

Unused token capacity doesn't cost money, but if the model tends to over-explain, a tight cap keeps outputs lean.

Real numbers

On one client project (200K API calls/month), applying these six techniques cut the monthly bill from $2,400 to $380 — an 84% reduction — with no measurable change in output quality.

Need to optimize an existing AI pipeline? Get in touch — we've done this for several teams.