A reusable procurement checklist for an internal evaluation, useful before any sales conversation.
Choosing an AI API looks like a technical decision. It is usually a procurement decision wearing a technical coat. The model is one line of your stack. The billing unit, the failure behavior, and the data terms shape your costs and your roadmap for a year.
Here are the ten questions that most reliably separate a provider that fits from one that merely demos well.
- Which endpoints do you actually call? Write down every model call your product makes: chat completions, audio transcriptions, embeddings, image generation. A provider that covers chat but not transcription means two vendors, two keys, and two invoices. Match the endpoint list before anything else.
- Is it OpenAI-compatible? If your code uses an OpenAI client, a compatible base URL and model name can move most of your calls without a rewrite. Ask what changes: usually the base URL, the key, and the model identifier. Keep the old path behind a flag so you can switch back.
- What is the billing unit? Tokens, requests, or minutes of audio. The unit decides which workload gets cheap and which gets expensive. A transcription-heavy product and an output-heavy chat product will not see the same bill from the same headline rate.
- Do the published rates match the shape of your traffic? If most of your tokens are input, the input rate dominates. If your assistant writes long answers, the output rate dominates. Estimate your monthly mix before you compare two providers, because the lower headline rate is often the more expensive one for your specific ratio. The cost calculator works from published rates, so you can test your own mix.
- What happens at the limit? Rate limits and timeouts are normal. Ask which status codes you get, whether retries are safe, and what backoff the provider expects. A provider that fails cleanly is worth more than one that fails silently.
- How is your data handled? Ask about retention, whether inputs train anything, and which sub-processors sit behind the API. If your customers are in a regulated industry, this answer is a gate, not a preference.
- How fast is it on your workload? Do not trust a benchmark page, including ours. Send your own prompts, from your own network, and measure the latency that matters to your users. A method for this is in our evaluation guide.
- How do you migrate and roll back? A safe move is a canary: send a small share of traffic to the new provider, compare output and error rates, and keep a one-line rollback. Ask the provider what they recommend. The migration playbook covers the wiring.
- Who supports you when it breaks? Find out whether you get an email desk, a Slack channel, or a named contact, and what the escalation path is at your volume. This is often the real difference between two providers with identical rates.
- What does the commercial relationship look like at your scale? Ask how pricing changes as volume grows, what commitments exist, and what the exit looks like. Get it in writing before it matters.
Score each answer from one to five for fit. A provider that scores well on one, two, and six is usually a safe first step. The rest can improve with time.
If you would rather map your own workload against these questions with an engineer, that is exactly what the session at https://webwright.ai/consulting is for. Thirty minutes, no slides.
The WebWright Team