Skip to main content

Streaming

Get responses in real-time as they're generated.

Enable Streaming​

Set stream=True (Python) or stream: true (JavaScript):

Python​

stream = client.chat.completions.create(
  model="bizora",
  messages=[{"role": "human", "content": "What is section 169?"}],
  stream=True
)

for chunk in stream:
  if chunk.choices[0].delta.content:
      print(chunk.choices[0].delta.content, end="", flush=True)

JavaScript​

const stream = await client.chat.completions.create({
model: 'bizora',
messages: [{ role: 'human', content: 'What is section 169?' }],
stream: true
});

for await (const chunk of stream) {
if (chunk.choices[0]?.delta?.content) {
  process.stdout.write(chunk.choices[0].delta.content);
}
}

Multi-Turn Conversations​

messages = [
  {"role": "human", "content": "What is section 169?"},
  {"role": "ai", "content": "Section 169 refers to..."},
  {"role": "human", "content": "What are the requirements?"}
]

stream = client.chat.completions.create(
  model="bizora",
  messages=messages,
  stream=True
)

Response Format​

Each chunk contains incremental content:

{
  "id": "chatcmpl-abc123",
  "object": "chat.completion.chunk",
  "created": 1677652288,
  "model": "bizora",
  "choices": [{
    "index": 0,
    "delta": {
      "content": "text chunk"
    },
    "finish_reason": null
  }]
}

The stream ends with:

data: [DONE]