# Chat Completions

`POST /chat/completions` generates the model's reply to a conversation. The server renders `messages` into a prompt for the model. Use it for chat and instruction-tuned models, reasoning, and tool calls.

The examples use the `client` and `MODEL` from the [Overview](https://tinker-docs.thinkingmachines.ai/inference/openai/overview/index.md):

```py
response = client.chat.completions.create(
    model=MODEL,
    messages=[{"role": "user", "content": "What is 17 * 23?"}],
)
print(response.choices[0].message.content)
```

## Reasoning effort

Not every model supports `reasoning_effort`. A value that the model does not take returns HTTP 400, and the error message lists the values that the model accepts.

- **Thinking Machines models**, such as Inkling, take a string or a number from `0.0` to `0.99`. The default is `0.9`. The strings map to numbers: `none` → `0.0`, `minimal` → `0.1`, `low` → `0.2`, `medium` → `0.7`, `high` → `0.9`, and `xhigh` and `max` → `0.99`.
- **Other models** take some of the strings, or `true` or `false` to turn reasoning on or off. When you leave it out, the model uses its own default. [Hybrid models](https://tinker-docs.thinkingmachines.ai/tinker/models/models_and_pricing/#model-types) can turn reasoning off; Reasoning models always reason.
- The OpenAI SDK types `reasoning_effort` as a string. It still sends a number or a boolean, but a type checker flags it; `extra_body={"reasoning_effort": 0.8}` avoids that.

```py
response = client.chat.completions.create(
    model=MODEL,
    messages=[{"role": "user", "content": "What is 17 * 23?"}],
    reasoning_effort="high",
)
```

## Streaming

Each chunk carries the next piece of the reply: reasoning in `reasoning_content`, or text in `content`. With `include_usage`, the last chunk also has `usage`.

```py
stream = client.chat.completions.create(
    model=MODEL,
    messages=[{"role": "user", "content": "What is 17 * 23?"}],
    stream=True,
    stream_options={"include_usage": True},
)
part = None
for chunk in stream:
    if chunk.choices:
        delta = chunk.choices[0].delta
        parts = {"Reasoning": delta.reasoning_content, "Answer": delta.content}
        for name, text in parts.items():
            if text:
                if name != part:
                    print(f"\n[{name}]")
                    part = name
                print(text, end="", flush=True)
    if chunk.usage:
        print("\nTokens:", chunk.usage.total_tokens)
```

## Tool calls

Tool calls need `tools` in the request and a chat model, not a [Base model](https://tinker-docs.thinkingmachines.ai/tinker/models/models_and_pricing/#model-types). If a model has no tool-call support, its calls come back as text in `content`. When the reply contains a tool call, `finish_reason` is `"tool_calls"`. Run the function yourself, then send its result back in a `tool` message.

```py
import json

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather in a city.",
            "parameters": {
                "type": "object",
                "properties": {"city": {"type": "string"}},
                "required": ["city"],
            },
        },
    }
]
messages = [{"role": "user", "content": "What's the weather in Paris?"}]
reply = client.chat.completions.create(model=MODEL, messages=messages, tools=tools).choices[0].message

if reply.tool_calls:  # None when the model answers with text only
    messages.append(
        {
            "role": "assistant",
            "content": reply.content,
            "reasoning_content": reply.reasoning_content,
            "tool_calls": [call.model_dump() for call in reply.tool_calls],
        }
    )
    for call in reply.tool_calls:  # the model can call more than one function
        args = json.loads(call.function.arguments)  # {"city": "Paris"}
        result = {"city": args["city"], "temp_c": 18}  # run your own function here
        messages.append({"role": "tool", "tool_call_id": call.id, "content": json.dumps(result)})
    response = client.chat.completions.create(model=MODEL, messages=messages, tools=tools)
```

Notes:

- We recommend that you leave `tool_choice` and `parallel_tool_calls` at their defaults. The server cannot force a model to follow them. See [Limits and errors](#chat-completions-limits).
- The server does not stop generation at a tool call. Generation ends at the model's end token, a `stop` sequence, or the token limit. Any text after the call comes back in `content`.

## Multi-turn conversations

Send each earlier assistant turn back with its `reasoning_content`. If you leave it out, the model sees the earlier turns without the reasoning that produced them. Some models ignore reasoning from earlier turns.

```py
messages = [{"role": "user", "content": "What is 17 * 23?"}]
reply = client.chat.completions.create(model=MODEL, messages=messages).choices[0].message

messages += [
    {
        "role": "assistant",
        "content": reply.content,
        "reasoning_content": reply.reasoning_content,
    },
    {"role": "user", "content": "Now add 9 to that."},
]
response = client.chat.completions.create(model=MODEL, messages=messages)
```

## Limits and errors

- **No structured output.** The server does not support `response_format` (JSON mode or JSON Schema). It ignores the field, and the reply is free text.
- **`tool_choice` and `parallel_tool_calls`.** Tinker serves many model families, so the server cannot guarantee that a model follows these settings. When you set them, the server checks each reply against them. A reply that breaks them fails the request with HTTP 422, for example when the model calls no function with `tool_choice: "required"`. We recommend that you leave both at their defaults (`"auto"` and `true`). A model with no tool-call support rejects `"required"` with HTTP 400, and the error message says so.
- **Replies in more than one part.** Some models produce a reply in several parts, for example reasoning, then text, then more reasoning. A Chat Completions message has one `reasoning_content` and one `content`, so the server joins the parts of each kind, in order. To work with the exact tokens that the model sampled, use [Completions](https://tinker-docs.thinkingmachines.ai/inference/openai/completions/#completions-token-ids) with `return_token_ids`.
- **Output that cannot be parsed.** The server parses each reply strictly in the model's output format. If a reply does not match that format, the server returns the whole reply as plain text in `content`. Any reasoning or tool call in that reply stays in the text, not in `reasoning_content` or `tool_calls`. The `tool_choice` check still applies: with `tool_choice: "required"`, a reply that comes back as plain text fails with HTTP 422. When streaming, the server switches to plain text at the point of the error.
- **Errors during streaming.** The HTTP status is already 200 when an error occurs mid-stream, so the stream ends with an error event instead: `data: {"error": {"message": "...", "type": "..."}}`. The OpenAI SDK raises `APIError` when it reads this event.

## API reference

### **Message object**

One message in a conversation. A [`/chat/completions`](#post-chat-completions) request sends the conversation in `messages`, and the model's reply comes back as `choices[].message` in the [chat completion object](#chat-completion-object).

**Fields:**

- **role** (*string*, required) – `system`, `user`, `assistant`, or `tool`.
- **content** (*string | array | null*) – A string, or an array of content parts such as `{"type": "text", "text": "..."}`.
- **reasoning_content** (*string*) – On an assistant message, the reasoning of that turn.
- **tool_calls** (*array*) – On an assistant message: the functions that the model called. Each item has an `id`, `"type": "function"`, and a `function` with a `name` and `arguments`. `arguments` is a JSON string.
- **tool_call_id** (*string*) – On a tool message: the `id` of the tool call whose result the message holds.

**Example:** a conversation with one tool call.

```json
[
  {"role": "system", "content": "You are a helpful assistant."},
  {"role": "user", "content": "What's the weather in Paris?"},
  {
    "role": "assistant",
    "content": null,
    "reasoning_content": "I need the current weather, so I will call get_weather.",
    "tool_calls": [
      {
        "id": "call_1",
        "type": "function",
        "function": {"name": "get_weather", "arguments": "{\"city\": \"Paris\"}"}
      }
    ]
  },
  {"role": "tool", "tool_call_id": "call_1", "content": "{\"temp_c\": 18}"},
  {"role": "assistant", "content": "It is 18 °C in Paris."}
]
```

### *POST* **/chat/completions**

Generate the model's reply to a conversation.

**Parameters:**

- **model** (*string*, required) – The model to sample. Its form depends on the base URL; see [Model](https://tinker-docs.thinkingmachines.ai/inference/openai/overview/#using-the-openai-client).
- **messages** (*array of [message objects](#chat-completions-message)*, required) – The conversation so far.
- **max_completion_tokens** (*integer*, optional) – The maximum number of tokens to generate. When unset, generation can run to the end of the model's context.
- **temperature** (*number*, default: `1`, minimum: `0`) – Higher values make the output more varied; lower values make it more predictable.
- **reasoning_effort** (*string | number | boolean*, optional) – How much the model reasons before it answers. See [Reasoning effort](#controlling-thinking-effort).
- **stream** (*boolean*, default: `false`) – Send the response in pieces as the model generates it, as server-sent events.
- **stream_options** (*object*, optional) – Set `include_usage` to `true` to get token counts on the last chunk.
- **tools** (*array*, optional) – Functions that the model can call. Each item is `{"type": "function", "function": {...}}`, where the function has a `name`, a `description`, and `parameters` as a JSON Schema. See [Tool calls](#chat-completions-tool-calls).
- **tool_choice** (*string | object*, optional) – `"none"`, `"auto"`, `"required"`, or one function that the model must call. We recommend the default, `"auto"`; see [Limits and errors](#chat-completions-limits).
- **parallel_tool_calls** (*boolean*, default: `true`) – Let the model call more than one tool in a turn. We recommend the default; see [Limits and errors](#chat-completions-limits).
- **stop** (*string | array of strings | array of integers*, optional) – Text or token IDs that end generation.
- **top_p** (*number*, default: `1`, range: `(0, 1]`) – Sample only from the most likely tokens whose probabilities add up to `top_p`. `1` keeps all tokens.
- **top_k** (*integer*, default: `-1`) – Sample only from the `k` most likely tokens. `-1` or `0` keeps all tokens. Tinker-specific.
- **seed** (*integer*, optional) – The random seed for sampling.
- **logprobs** (*boolean*, default: `false`) – Return the log probability of each generated token.
- **top_logprobs** (*integer*, optional, range: `[0, 20]`) – For each generated token, also return this many of the most likely alternatives. Needs `logprobs`.

The server ignores fields that this page does not list.

The model's reasoning always comes back in `reasoning_content`. A Tinker-specific option, `separate_reasoning: false`, puts it into `content` instead; we do not recommend it.

**Returns:**

- A [chat completion object](#chat-completion-object). With `stream`, a series of [chat completion chunks](#chat-completion-chunk-object).

**Example request:**

```py
response = client.chat.completions.create(
    model=MODEL,
    messages=[{"role": "user", "content": "What is 17 * 23?"}],
    max_completion_tokens=256,
    temperature=0.7,
)
print(response.choices[0].message.reasoning_content)
print(response.choices[0].message.content)
```

**Example response:**

```json
{
  "id": "chatcmpl-3f9a1c2e",
  "object": "chat.completion",
  "created": 1794500000,
  "model": "thinkingmachines/Inkling:peft:262144",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "17 * 23 = 391.",
        "reasoning_content": "17 * 20 = 340, and 17 * 3 = 51. 340 + 51 = 391.",
        "name": null,
        "tool_calls": null,
        "tool_call_id": null
      },
      "finish_reason": "stop",
      "logprobs": null
    }
  ],
  "usage": {
    "prompt_tokens": 16,
    "completion_tokens": 48,
    "total_tokens": 64,
    "prompt_tokens_details": {"cached_tokens": 0}
  }
}
```

### **Chat completion object**

The response of a [`/chat/completions`](#post-chat-completions) request.

**Fields:**

- **id** (*string*) – The ID of the completion.
- **object** (*string*) – Always `"chat.completion"`.
- **created** (*integer*) – When the completion was created, in Unix seconds.
- **model** (*string*) – The `model` from the request.
- **choices** (*array*) – A list with one item, the reply. The item has:
  - **index** (*integer*) – Always `0`.
  - **message** (*[message object](#chat-completions-message)*) – The reply: `role`, `content`, `reasoning_content`, and `tool_calls`.
  - **finish_reason** (*string*) – Why generation stopped (`"stop"` or `"length"`), or `"tool_calls"` when the reply contains a tool call.
  - **logprobs** (*object | null*) – When `logprobs` is `true`: `content`, a list with one item per generated token (`token`, `logprob`, `bytes`, `top_logprobs`).
- **usage** (*object*) – Token counts: `prompt_tokens`, `completion_tokens`, `total_tokens`, and `prompt_tokens_details.cached_tokens`.

### **Chat completion chunk**

One piece of a streamed [`/chat/completions`](#post-chat-completions) response. It has the fields of a [chat completion object](#chat-completion-object), with these differences:

- **object** (*string*) – Always `"chat.completion.chunk"`.
- **choices[].delta** (*object*) – Takes the place of `message`. It holds only the new reasoning, text, or tool calls of this piece.
- **choices[].finish_reason** (*string | null*) – `null` until the last chunk.
- **usage** (*object | null*) – Only on the last chunk, and only when `stream_options.include_usage` is `true`.