45 min
Lesson 3 of 6
Streaming & Real-Time Responses
From the course OpenAI API Masterclass
Streaming API Responses
Streaming delivers tokens as they're generated, dramatically improving perceived latency for users.
Basic Streaming
const stream = await openai.chat.completions.create({
model: "gpt-4o",
messages: [{ role: "user", content: "Write a poem about AI" }],
stream: true,
});
for await (const chunk of stream) {
const content = chunk.choices[0]?.delta?.content || "";
process.stdout.write(content);
}
Server-Sent Events (SSE)
For web applications, stream responses via SSE:
// Express endpoint
app.post("/api/chat", async (req, res) => {
res.setHeader("Content-Type", "text/event-stream");
res.setHeader("Cache-Control", "no-cache");
const stream = await openai.chat.completions.create({
model: "gpt-4o",
messages: req.body.messages,
stream: true,
});
for await (const chunk of stream) {
const data = JSON.stringify(chunk);
res.write(`data: ${data}\n\n`);
}
res.write("data: [DONE]\n\n");
res.end();
});
Stream Processing Patterns
- Token counting: Track tokens in real-time from stream usage data
- Abort handling: Use AbortController to cancel long-running streams
- Error recovery: Catch mid-stream errors and provide fallback content
- Backpressure: Buffer tokens if the client can't consume fast enough
Stream Helpers
The SDK provides helper methods for common streaming patterns:
const runner = openai.beta.chat.completions.stream({
model: "gpt-4o",
messages: [{ role: "user", content: "Hello" }],
});
runner.on("content", (delta) => process.stdout.write(delta));
const finalMessage = await runner.finalMessage();
Key Takeaways
- •Streaming reduces perceived latency by showing tokens as they generate
- •Use SSE for web applications and async iterators for server-side
- •Implement abort handling and error recovery for production streams
- •SDK stream helpers simplify event handling and final message capture
0 of 6 lessons marked complete
Progress is kept in this browser only.