Streaming, timeouts, and cancellation

Check streaming support in the public model catalog. Set stream: true for Chat, Responses, and Messages; Gemini uses :streamGenerateContent?alt=sse without a body-level stream field. Start consuming only after a successful HTTP status and a text/event-stream content type.

Protocol	Normal completion
Chat Completions	data: [DONE]
Responses	response.completed; also handle failed or incomplete terminal events
Messages	message_stop
Gemini	Consume generation events and check candidate finish reasons and stream completion

Blank lines delimit SSE events, not network chunks. An event can have multiple data: lines, and UTF-8 characters can span chunks. Do not require every protocol to emit [DONE]. In-stream errors, failed events, and incomplete events are not successful generation results.

Official SDK

TypeScript 0.3.0 exposes streamChatCompletion, streamResponse, streamMessage, and streamGeminiContent as async event iterables. Events preserve event, raw data, and optional id; callers parse JSON and identify terminal/error events for their protocol.

const controller = new AbortController()
const timeout = setTimeout(() => controller.abort(), 60_000)
try {
  for await (const event of gateway.streamResponse(
    { model, input: '<YOUR_PROMPT>' },
    { signal: controller.signal },
  )) {
    // Update the UI using event.event and JSON.parse(event.data); do not log bodies.
  }
} finally {
  clearTimeout(timeout)
}

Exiting iteration closes the response stream; AbortSignal cancels pending network requests. The SDK does not retry or replay streams automatically. Each SSE event is limited to 1,048,576 UTF-16 code units; larger events fail explicitly. Swift/Kotlin currently buffer JSON responses and reject stream: true before sending; native apps needing streaming must use an SSE-capable transport implementation.

Set connection, first-byte, and overall deadlines appropriate to your workload; the example supplies only an overall deadline. Swift/Kotlin offer a request timeout through GatewayRequestOptions.timeout. Usage already generated may still be charged after cancellation, with remaining reservations released under platform rules. Disconnecting does not imply zero cost. Retain X-Request-Id, but never log request or response bodies.

Chinese reasoning and tool deltas

Chat streams from DeepSeek, Kimi, MiniMax or GLM may contain delta.reasoning_content; accumulate it separately from final delta.content. Tool arguments may span events. Assemble fragments by tool_calls index before parsing JSON, and retain reasoning and tool messages for the next turn. Parameters and protocols depend on the selected model. See Chinese model integration.