A chat API does not remember your conversation. Each call is stateless: the model sees only the messages array you send in that request. If your second turn sends only the new user message, the model has no idea what you asked a moment ago. Conversation memory is something your client builds by resending the history, and that is also why cost creeps up over a long thread.

The whole history goes in every request

The chat endpoint takes a list of messages with roles. You seed it with a system message, then append the user turn, then append the assistant reply you got back.

from openai import OpenAI

client = OpenAI(base_url="https://webwright.ai/v1", api_key="www_v1_YOUR_API_KEY")

messages = [{"role": "system", "content": "You are a concise assistant."}]

def send(user_text):
    messages.append({"role": "user", "content": user_text})
    completion = client.chat.completions.create(
        model="wren-r1",
        messages=messages,
        max_tokens=512,
    )
    reply = completion.choices[0].message.content
    messages.append({"role": "assistant", "content": reply})
    return reply, completion.usage

Every call sends the entire messages list, not just the newest line. The model stays consistent because it can see the earlier turns, and the usage object tells you the price of that consistency.

Watch the prompt grow

The response includes usage.prompt_tokens, usage.completion_tokens and usage.total_tokens. Log prompt_tokens per turn and you will see it climb: turn one is a system line and one question; turn ten is the system line plus nine exchanges. You pay for the prompt on every request, so the same history is billed again each turn. That is the number to watch, not just the total.

Three ways to bound the history

Pick the one that matches the feature. None of them is universal.

  1. Keep a window. Keep the system message and the last N turns, and drop the oldest pairs. Simple, predictable, and it loses old detail.
  2. Summarize older turns. When the thread passes a threshold, ask the model once to compress everything before the window into a short recap, then keep that recap plus the recent turns. More tokens up front, fewer later.
  3. Drop noise. Long tool outputs, pasted logs and dead-end branches are the usual bloat. Keep the decisions and drop the scaffolding.

A small helper makes the window explicit:

def trim(messages, keep_turns=8):
    system = [m for m in messages if m["role"] == "system"]
    rest = [m for m in messages if m["role"] != "system"]
    return system + rest[-2 * keep_turns:]

Call it before each request. The system message stays first; the newest turns stay; the rest falls away. Because the model field is currently advisory on this API, you do not manage a per-model context budget yet, but the trimming logic is the same code you will want when more models arrive.

A short checklist

  • Resend the full history each turn; the server holds no state.
  • Log usage.prompt_tokens per turn, not only totals.
  • Bound history with a window, a summary, or noise removal before it becomes a cost problem.
  • Keep the system message pinned first so behavior stays stable as the window slides.

Conversation memory is a client concern, and once you treat it that way the design is small: append, measure, trim.

Want a second set of eyes on the token growth of your assistant feature? Book a short consult and we will walk the request path with you.

The WebWright Team