Skip to content
View as Markdown

Completions

POST /completions continues a prompt as it is; the server does not render it. Use it for pretrained base models, plain text continuation, and prompts that you render to token IDs yourself (see the Rendering tutorial).

The examples use the client from the Overview:

response = client.completions.create(
    model="Qwen/Qwen3.5-9B-Base",
    prompt="The capital of France is",
    max_tokens=50,
)
print(response.choices[0].text)

Token IDs in and out

Send token IDs as prompt, and set return_token_ids to get the generated token IDs back. Use this when you render prompts yourself, or to send sampled tokens into the next prompt without tokenizing text again.

response = client.completions.create(
    model="Qwen/Qwen3.5-9B-Base",
    prompt=tokens,  # token IDs from your own renderer or the model's tokenizer
    max_tokens=50,
    extra_body={"return_token_ids": True},
)
choice = response.choices[0]
print(choice.token_ids)         # the generated token IDs
print(choice.prompt_token_ids)  # the prompt token IDs

Images

To send images to a vision model, use prompt_chunks instead of prompt. It lists the prompt in order, as token parts and image parts. The server does not render the prompt, so your token parts must contain the tokens that the model's renderer puts around each image; the server checks this and rejects a prompt without them. Each image is a base64 data URL (PNG, JPEG, GIF, or WebP; a GIF uses only its first frame). Models that do not take images return HTTP 400.

The OpenAI SDK has no prompt_chunks argument, so send the JSON yourself:

from os import getenv

import httpx

body = {
    "model": "thinkingmachines/Inkling:peft:262144",
    "prompt_chunks": [
        {"type": "tokens", "tokens": before_image},  # up to and including the image start tokens
        {"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}},
        {"type": "tokens", "tokens": after_image},  # the image end tokens and the rest of the prompt
    ],
    "max_tokens": 100,
}
response = httpx.post(
    "https://tinker.thinkingmachines.dev/services/tinker-prod/oai/api/v1/completions",
    headers={"Authorization": f"Bearer {getenv('TINKER_API_KEY')}"},
    json=body,
)
print(response.json()["choices"][0]["text"])

With prompt_chunks, the response has no prompt_token_ids, because image positions have no token IDs.

API reference

POST /completions

Continue a prompt.

Parameters:

  • model (string, required) – The model to sample. Its form depends on the base URL; see Model.
  • prompt (string | array of integers) – One prompt, as text or as token IDs. Set either prompt or prompt_chunks. The server tokenizes text with the model's tokenizer, and uses token IDs as they are. An array that holds one prompt also works; more than one prompt returns HTTP 400.
  • prompt_chunks (array) – A prompt with images: token parts {"type": "tokens", "tokens": [...]} and image parts {"type": "image_url", "image_url": {"url": "..."}}, in prompt order. See Images. Tinker-specific.
  • max_tokens (integer, optional) – The maximum number of tokens to generate. When unset, generation can run to the end of the model's context.
  • temperature, stream, stream_options, stop, top_p, top_k, seed – Same as for /chat/completions.
  • return_token_ids (boolean, default: false) – Also return token_ids and prompt_token_ids on each choice, so you can send the sampled tokens back as the next prompt. Tinker-specific.
  • logprobs (integer, optional, range: [0, 20]) – Return the log probability of each generated token, and this many of the most likely alternatives at each position, plus the generated token.

Returns:

  • A completion object. With stream, a series of completion objects, each with the next piece of text.

Example request:

response = client.completions.create(
    model="Qwen/Qwen3.5-9B-Base",
    prompt="The capital of France is",
    max_tokens=50,
)
print(response.choices[0].text)

Example response:

{
  "id": "cmpl-8d2b4e7f",
  "object": "text_completion",
  "created": 1794500000,
  "model": "Qwen/Qwen3.5-9B-Base",
  "choices": [
    {
      "index": 0,
      "text": " Paris. It is also the largest city in France.",
      "finish_reason": "stop",
      "logprobs": null
    }
  ],
  "usage": {
    "prompt_tokens": 5,
    "completion_tokens": 11,
    "total_tokens": 16,
    "prompt_tokens_details": {"cached_tokens": 0}
  }
}

Completion object

The response of a /completions request.

Fields:

  • id, created, model, usage – Same as in the chat completion object.
  • object (string) – Always "text_completion".
  • choices (array) – A list with one item, the generated text. The item has:
    • index (integer) – Always 0.
    • text (string) – The generated text.
    • finish_reason (string) – Why generation stopped: "stop" or "length".
    • logprobs (object | null) – When logprobs is set: tokens, token_logprobs, top_logprobs, and text_offset.
    • token_ids (array of integers) – When return_token_ids is true: the generated token IDs.
    • prompt_token_ids (array of integers) – When return_token_ids is true: the prompt token IDs. When streaming, only the first chunk has it. Not returned for a prompt_chunks prompt.