When AI should stay out of the calculation
AI is useful when the work depends on language, classification, retrieval, or context. It is usually the wrong tool for a calculation or rule that must produce the same answer every time. Strong production systems use both, with a deliberate boundary between them.
01
Use conventional software where correctness is definable
Tax rates, engineering formulas, permission checks, invoice totals, validation rules, and pricing thresholds can be expressed deterministically. They belong in code that can be tested against known inputs and expected outputs.
Asking a language model to reproduce deterministic logic adds variability without adding value. Even when the answer looks right in a demonstration, there is no reason to accept uncertainty where a testable function can provide a guarantee.
02
Use AI where the input resists fixed rules
Emails, documents, questions, descriptions, and inconsistent human language rarely fit a complete rules engine. AI can classify them, extract proposed structure, retrieve relevant knowledge, and decide which deterministic tool should run next.
The model does not need to own the entire workflow. Its job may be as narrow as turning an unstructured request into validated fields that conventional software can process safely.
03
Make the boundary visible in the architecture
A production design should show which outputs came from a model, which came from code, and which were confirmed by a person. That distinction supports testing, troubleshooting, and sensible fallbacks when a model or provider is unavailable.
It also prevents a common failure mode: gradually moving more decisions into the prompt because changing prompt text feels faster than maintaining the underlying system.
04
Evaluate the interpretation, test the calculation
AI behavior needs representative evaluation cases and acceptable error boundaries. Deterministic behavior needs unit and integration tests. Treating both as prompt quality leaves the most important guarantees implicit.
The resulting system may still feel like one assistant to the user. Internally, it is a controlled combination of model judgment, explicit tools, fixed rules, and human authority.