Token Pricing Math

1. Concept Overview & Systemic Problem

When a newcomer looks at the pricing page of OpenAI or Anthropic, they see strange numbers:

• Input: $2.50 per 1M tokens
• Output: $10.00 per 1M tokens.

What do these "1M tokens" mean? How much will I pay if 100 of my clients ask the bot questions? Will an unexpected bill at the end of the month break me?

Token Pricing Math is the fundamental arithmetic of any AI project:

• By understanding this formula, you can calculate the cost of each transaction down to the cent.
• You will learn how to reduce costs by a factor of ten without sacrificing response quality.

The main principle for developers: the ability to translate billing report numbers into real dollars and cents in your wallet.

2. Cost Calculation Formula for a Single Request

┌─────────────────────────────────────────────────────────────┐
│                 COST FORMULA FOR A REQUEST                  │
├─────────────────────────────────────────────────────────────┤
│ Total Cost = (Input Tokens * Input Price)                   │
│                + (Output Tokens * Output Price)             │
├─────────────────────────────────────────────────────────────┤
│ EXAMPLE FOR GPT-4o-mini MODEL:                               │
│   • You sent an instruction file of 2,000 tokens             │
│   • The model generated a response of 500 tokens             │
│                                                             │
│ 1. Input:  2,000 * ($0.15 / 1,000,000)  = $0.00030          │
│ 2. Output:   500 * ($0.60 / 1,000,000)  = $0.00030          │
│ ─────────────────────────────────────────────────────────── │
│ 🎯 TOTAL: $0.0006 (Six hundred-thousandths of a dollar per request!) │
│ You can conduct over 1,600 such detailed dialogues for $1!  │
└─────────────────────────────────────────────────────────────┘

3. Four Rules for Budget Optimization in Projects

1. Router Rule: Send 80% of simple daily questions (greetings, product searches, text summaries) to inexpensive micro-models (GPT-4o-mini or Gemini Flash). Only call the flagship Claude 3.5 Sonnet for 20% of complex logical tasks.
2. Use Prompt Caching: If you have a large static system prompt, it is cached by the server and costs 90% less!
3. Limit Output Tokens (max_tokens): Never allow the model to write lengthy essays where a single number is needed.
4. Clear Chat History: Do not send all 40 previous messages if the client has already changed the topic of conversation.

4. Production Engineering Scenarios

01. Cost Management for High-Volume Queries

Implement a system that automatically routes common queries to low-cost models while reserving high-cost models for complex requests, ensuring budget adherence.

02. Dynamic Token Limiting

Develop a mechanism that dynamically adjusts the max_tokens parameter based on the context of the conversation, preventing unnecessary costs from lengthy outputs.

03. Effective Prompt Caching Strategies

Utilize caching strategies for static prompts to significantly reduce costs, ensuring that repeated queries leverage cached responses rather than incurring new processing fees.

5. Pitfalls, Common Mistakes & Security

Avoid overlooking the cumulative costs associated with high-frequency queries, which can lead to unexpected bills. Ensure that token limits are enforced strictly to prevent runaway costs from excessive output generation. Regularly audit API usage to identify and rectify inefficiencies in token consumption.