Skip to content
View as Markdown

OpenAI-Compatible API (Beta)

OpenAI-compatible inference lets you sample from any Tinker base model or checkpoint through two endpoints that work like the OpenAI API: /chat/completions and /completions. It is designed for testing your models, and you can also use it for production traffic:

  • Fast feedback: Sample from a checkpoint as soon as training saves it, while the training run keeps going.
  • Testing and evaluation: Run evaluations, scripts, and your own tools against your models.
  • Production traffic: Serve your models to your own users. For higher or more stable throughput, or guidance on large-scale setups, join our waitlist for production inference.

Using the OpenAI client

Point any OpenAI SDK at https://tinker.thinkingmachines.dev/services/tinker-prod/oai/api/v1 and authenticate with your Tinker API key. Set model to one of these:

  • A base model: its Tinker ID from Models & Pricing, such as thinkingmachines/Inkling:peft:262144.
  • Your checkpoint: the path of a checkpoint saved for sampling, such as tinker://…/sampler_weights/000080. It runs on the base model it was trained on.
  • Your checkpoint with a different context length: <base model ID>;<checkpoint path>, such as thinkingmachines/Inkling:peft:262144;tinker://…/sampler_weights/000080. The base model must be the model that the checkpoint was trained on, with the context length you want.
from os import getenv
from openai import OpenAI

MODEL = "thinkingmachines/Inkling:peft:262144"

client = OpenAI(
    base_url="https://tinker.thinkingmachines.dev/services/tinker-prod/oai/api/v1",
    api_key=getenv("TINKER_API_KEY"),
)

response = client.chat.completions.create(
    model=MODEL,
    messages=[{"role": "user", "content": "What is 17 * 23?"}],
)
print(response.choices[0].message.content)

Use Chat Completions for chat and instruction-tuned models. Use Completions for pretrained base models, or to send your own token IDs.

Sending requests without the OpenAI SDK

Send JSON to the same URLs, with your Tinker API key as a bearer token:

curl https://tinker.thinkingmachines.dev/services/tinker-prod/oai/api/v1/chat/completions \
  -H "Authorization: Bearer $TINKER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "thinkingmachines/Inkling:peft:262144",
    "messages": [{"role": "user", "content": "What is 17 * 23?"}],
    "reasoning_effort": "high"
  }'

extra_body is an option of the OpenAI SDK, not a field of the HTTP API: the SDK merges its keys into the top level of the request. In raw JSON, put fields such as top_k and reasoning_effort at the top level. The server ignores unknown fields with no error, so anything inside an extra_body key is lost.