View as Markdown

Chat Completions

POST /chat/completions generates the model's reply to a conversation. The server renders messages into a prompt for the model. Use it for chat and instruction-tuned models, reasoning, and tool calls.

The examples use the client and MODEL from the Overview:

response = client.chat.completions.create(
    model=MODEL,
    messages=[{"role": "user", "content": "What is 17 * 23?"}],
)
print(response.choices[0].message.content)

Reasoning effort

Not every model supports reasoning_effort. A value that the model does not take returns HTTP 400, and the error message lists the values that the model accepts.

Thinking Machines models, such as Inkling, take a string or a number from 0.0 to 0.99. The default is 0.9. The strings map to numbers: none → 0.0, minimal → 0.1, low → 0.2, medium → 0.7, high → 0.9, and xhigh and max → 0.99.
Other models take some of the strings, or true or false to turn reasoning on or off. When you leave it out, the model uses its own default. Hybrid models can turn reasoning off; Reasoning models always reason.
The OpenAI SDK types reasoning_effort as a string. It still sends a number or a boolean, but a type checker flags it; extra_body={"reasoning_effort": 0.8} avoids that.

response = client.chat.completions.create(
    model=MODEL,
    messages=[{"role": "user", "content": "What is 17 * 23?"}],
    reasoning_effort="high",
)

Streaming

Each chunk carries the next piece of the reply: reasoning in reasoning_content, or text in content. With include_usage, the last chunk also has usage.

stream = client.chat.completions.create(
    model=MODEL,
    messages=[{"role": "user", "content": "What is 17 * 23?"}],
    stream=True,
    stream_options={"include_usage": True},
)
part = None
for chunk in stream:
    if chunk.choices:
        delta = chunk.choices[0].delta
        parts = {"Reasoning": delta.reasoning_content, "Answer": delta.content}
        for name, text in parts.items():
            if text:
                if name != part:
                    print(f"\n[{name}]")
                    part = name
                print(text, end="", flush=True)
    if chunk.usage:
        print("\nTokens:", chunk.usage.total_tokens)

Tool calls

Tool calls need tools in the request and a chat model, not a Base model. If a model has no tool-call support, its calls come back as text in content. When the reply contains a tool call, finish_reason is "tool_calls". Run the function yourself, then send its result back in a tool message.

import json

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather in a city.",
            "parameters": {
                "type": "object",
                "properties": {"city": {"type": "string"}},
                "required": ["city"],
            },
        },
    }
]
messages = [{"role": "user", "content": "What's the weather in Paris?"}]
reply = client.chat.completions.create(model=MODEL, messages=messages, tools=tools).choices[0].message

if reply.tool_calls:  # None when the model answers with text only
    messages.append(
        {
            "role": "assistant",
            "content": reply.content,
            "reasoning_content": reply.reasoning_content,
            "tool_calls": [call.model_dump() for call in reply.tool_calls],
        }
    )
    for call in reply.tool_calls:  # the model can call more than one function
        args = json.loads(call.function.arguments)  # {"city": "Paris"}
        result = {"city": args["city"], "temp_c": 18}  # run your own function here
        messages.append({"role": "tool", "tool_call_id": call.id, "content": json.dumps(result)})
    response = client.chat.completions.create(model=MODEL, messages=messages, tools=tools)

Notes:

We recommend that you leave tool_choice and parallel_tool_calls at their defaults. The server cannot force a model to follow them. See Limits and errors.
The server does not stop generation at a tool call. Generation ends at the model's end token, a stop sequence, or the token limit. Any text after the call comes back in content.

Multi-turn conversations

Send each earlier assistant turn back with its reasoning_content. If you leave it out, the model sees the earlier turns without the reasoning that produced them. Some models ignore reasoning from earlier turns.

messages = [{"role": "user", "content": "What is 17 * 23?"}]
reply = client.chat.completions.create(model=MODEL, messages=messages).choices[0].message

messages += [
    {
        "role": "assistant",
        "content": reply.content,
        "reasoning_content": reply.reasoning_content,
    },
    {"role": "user", "content": "Now add 9 to that."},
]
response = client.chat.completions.create(model=MODEL, messages=messages)

Limits and errors

No structured output. The server does not support response_format (JSON mode or JSON Schema). It ignores the field, and the reply is free text.
tool_choice and parallel_tool_calls. Tinker serves many model families, so the server cannot guarantee that a model follows these settings. When you set them, the server checks each reply against them. A reply that breaks them fails the request with HTTP 422, for example when the model calls no function with tool_choice: "required". We recommend that you leave both at their defaults ("auto" and true). A model with no tool-call support rejects "required" with HTTP 400, and the error message says so.
Replies in more than one part. Some models produce a reply in several parts, for example reasoning, then text, then more reasoning. A Chat Completions message has one reasoning_content and one content, so the server joins the parts of each kind, in order. To work with the exact tokens that the model sampled, use Completions with return_token_ids.
Output that cannot be parsed. The server parses each reply strictly in the model's output format. If a reply does not match that format, the server returns the whole reply as plain text in content. Any reasoning or tool call in that reply stays in the text, not in reasoning_content or tool_calls. The tool_choice check still applies: with tool_choice: "required", a reply that comes back as plain text fails with HTTP 422. When streaming, the server switches to plain text at the point of the error.
Errors during streaming. The HTTP status is already 200 when an error occurs mid-stream, so the stream ends with an error event instead: data: {"error": {"message": "...", "type": "..."}}. The OpenAI SDK raises APIError when it reads this event.

API reference

Message object

One message in a conversation. A /chat/completions request sends the conversation in messages, and the model's reply comes back as choices[].message in the chat completion object.

Fields:

role (string, required) – system, user, assistant, or tool.
content (string | array | null) – A string, or an array of content parts such as {"type": "text", "text": "..."}.
reasoning_content (string) – On an assistant message, the reasoning of that turn.
tool_calls (array) – On an assistant message: the functions that the model called. Each item has an id, "type": "function", and a function with a name and arguments. arguments is a JSON string.
tool_call_id (string) – On a tool message: the id of the tool call whose result the message holds.

Example: a conversation with one tool call.

[
  {"role": "system", "content": "You are a helpful assistant."},
  {"role": "user", "content": "What's the weather in Paris?"},
  {
    "role": "assistant",
    "content": null,
    "reasoning_content": "I need the current weather, so I will call get_weather.",
    "tool_calls": [
      {
        "id": "call_1",
        "type": "function",
        "function": {"name": "get_weather", "arguments": "{\"city\": \"Paris\"}"}
      }
    ]
  },
  {"role": "tool", "tool_call_id": "call_1", "content": "{\"temp_c\": 18}"},
  {"role": "assistant", "content": "It is 18 °C in Paris."}
]

POST /chat/completions

Generate the model's reply to a conversation.

Parameters:

model (string, required) – The model to sample. Its form depends on the base URL; see Model.
messages (array of message objects, required) – The conversation so far.
max_completion_tokens (integer, optional) – The maximum number of tokens to generate. When unset, generation can run to the end of the model's context.
temperature (number, default: 1, minimum: 0) – Higher values make the output more varied; lower values make it more predictable.
reasoning_effort (string | number | boolean, optional) – How much the model reasons before it answers. See Reasoning effort.
stream (boolean, default: false) – Send the response in pieces as the model generates it, as server-sent events.
stream_options (object, optional) – Set include_usage to true to get token counts on the last chunk.
tools (array, optional) – Functions that the model can call. Each item is {"type": "function", "function": {...}}, where the function has a name, a description, and parameters as a JSON Schema. See Tool calls.
tool_choice (string | object, optional) – "none", "auto", "required", or one function that the model must call. We recommend the default, "auto"; see Limits and errors.
parallel_tool_calls (boolean, default: true) – Let the model call more than one tool in a turn. We recommend the default; see Limits and errors.
stop (string | array of strings | array of integers, optional) – Text or token IDs that end generation.
top_p (number, default: 1, range: (0, 1]) – Sample only from the most likely tokens whose probabilities add up to top_p. 1 keeps all tokens.
top_k (integer, default: -1) – Sample only from the k most likely tokens. -1 or 0 keeps all tokens. Tinker-specific.
seed (integer, optional) – The random seed for sampling.
logprobs (boolean, default: false) – Return the log probability of each generated token.
top_logprobs (integer, optional, range: [0, 20]) – For each generated token, also return this many of the most likely alternatives. Needs logprobs.

The server ignores fields that this page does not list.

The model's reasoning always comes back in reasoning_content. A Tinker-specific option, separate_reasoning: false, puts it into content instead; we do not recommend it.

Returns:

A chat completion object. With stream, a series of chat completion chunks.

Example request:

response = client.chat.completions.create(
    model=MODEL,
    messages=[{"role": "user", "content": "What is 17 * 23?"}],
    max_completion_tokens=256,
    temperature=0.7,
)
print(response.choices[0].message.reasoning_content)
print(response.choices[0].message.content)

Example response:

{
  "id": "chatcmpl-3f9a1c2e",
  "object": "chat.completion",
  "created": 1794500000,
  "model": "thinkingmachines/Inkling:peft:262144",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "17 * 23 = 391.",
        "reasoning_content": "17 * 20 = 340, and 17 * 3 = 51. 340 + 51 = 391.",
        "name": null,
        "tool_calls": null,
        "tool_call_id": null
      },
      "finish_reason": "stop",
      "logprobs": null
    }
  ],
  "usage": {
    "prompt_tokens": 16,
    "completion_tokens": 48,
    "total_tokens": 64,
    "prompt_tokens_details": {"cached_tokens": 0}
  }
}

Chat completion object

The response of a /chat/completions request.

Fields:

id (string) – The ID of the completion.
object (string) – Always "chat.completion".
created (integer) – When the completion was created, in Unix seconds.
model (string) – The model from the request.
choices (array) – A list with one item, the reply. The item has:
- index (integer) – Always 0.
- message (message object) – The reply: role, content, reasoning_content, and tool_calls.
- finish_reason (string) – Why generation stopped ("stop" or "length"), or "tool_calls" when the reply contains a tool call.
- logprobs (object | null) – When logprobs is true: content, a list with one item per generated token (token, logprob, bytes, top_logprobs).
usage (object) – Token counts: prompt_tokens, completion_tokens, total_tokens, and prompt_tokens_details.cached_tokens.

Chat completion chunk

One piece of a streamed /chat/completions response. It has the fields of a chat completion object, with these differences:

object (string) – Always "chat.completion.chunk".
choices[].delta (object) – Takes the place of message. It holds only the new reasoning, text, or tool calls of this piece.
choices[].finish_reason (string | null) – null until the last chunk.
usage (object | null) – Only on the last chunk, and only when stream_options.include_usage is true.