Rate limits and concurrency

Effective limits combine current plan entitlements and API-key restrictions, including requests per minute (RPM), tokens per minute (TPM), and concurrency. Do not hard-code values that are not published.

After rate_limited / HTTP 429, stop immediate replay, wait for Retry-After when present, and use exponential backoff, jitter, and a bounded queue. Persistent 429 responses call for lower send rate or concurrency, not more retry workers.

Use /v1/models and /v1/balance for connectivity checks without starting a model request. Production monitoring should distinguish authentication, entitlement, balance, and rate-limit failures.