LLM Cost Optimization: Your Bill Is an Architecture Problem, Not a Prompt Problem

You launch an LLM feature. Users like it. Usage grows. Someone adds conversation history. Someone else adds RAG. Then tools. Then retries. Then an evaluator that uses another LLM. A few months later, finance asks a question engineering…