Streaming Responses Made Easy
Users perceive streaming responses as much faster. The first token often arrives in under a second, even when the full answer takes longer.
Enable stream in the request
Add "stream": true to your request body. The server then returns server-sent events with one delta per chunk.
Event shape
| Field | Meaning |
|---|---|
| choices[0].delta.content | The next piece of text |
| choices[0].finish_reason | stop / length / tool_calls |
| usage | Final token counts |
Handle the final usage event 📈
The last event includes usage. Accumulate it so you can display tokens consumed to your users.
Common pitfalls
- Do not buffer the whole stream
- Handle network reconnects
- Cancel the request when the user stops
With streaming enabled your app instantly feels snappier ✨.
Frequently asked questions ❓
Do I still get the full usage numbers?
Yes. The final chunk of the stream includes the usage object with input and output token counts.
Can I stream with the official SDKs?
Yes. Set stream=True in openai-python (or stream in openai-node) and iterate over the deltas.
Do tool calls work in streaming mode?
Yes. Tool-call deltas arrive in the same server-sent event stream as text deltas.
Does streaming cost more?
No. Streaming uses the same per-token pricing as non-streaming requests.
Comments (0)