Skip to contentSkip to content

Streaming, timeouts, and cancellation

Set stream: true on supported requests. SSE delivers data: {JSON} events and ends with data: [DONE]. Parse complete event boundaries rather than treating arbitrary network chunks as JSON.

Use separate connection, first-byte, and whole-request timeouts. Start consuming only after an HTTP success with the expected streaming content type. Errors before streaming use the standard error body; errors after streaming begins arrive as stream events before closure.

On cancellation, close the response body and downstream work. Generated usage is still settled and the unused reservation is released; a disconnected client does not imply zero cost. Retain X-Request-Id and do not log prompts or responses.