AI Model Wiki

    Skip to main content
    45 min
    Lesson 3 of 6

    Streaming & Real-Time Responses

    From the course OpenAI API Masterclass

    Streaming API Responses

    Streaming delivers tokens as they're generated, dramatically improving perceived latency for users.

    Basic Streaming

    const stream = await openai.chat.completions.create({
      model: "gpt-4o",
      messages: [{ role: "user", content: "Write a poem about AI" }],
      stream: true,
    });
    
    for await (const chunk of stream) {
      const content = chunk.choices[0]?.delta?.content || "";
      process.stdout.write(content);
    }
    

    Server-Sent Events (SSE)

    For web applications, stream responses via SSE:

    // Express endpoint
    app.post("/api/chat", async (req, res) => {
      res.setHeader("Content-Type", "text/event-stream");
      res.setHeader("Cache-Control", "no-cache");
    
      const stream = await openai.chat.completions.create({
        model: "gpt-4o",
        messages: req.body.messages,
        stream: true,
      });
    
      for await (const chunk of stream) {
        const data = JSON.stringify(chunk);
        res.write(`data: ${data}\n\n`);
      }
      res.write("data: [DONE]\n\n");
      res.end();
    });
    

    Stream Processing Patterns

    • Token counting: Track tokens in real-time from stream usage data
    • Abort handling: Use AbortController to cancel long-running streams
    • Error recovery: Catch mid-stream errors and provide fallback content
    • Backpressure: Buffer tokens if the client can't consume fast enough

    Stream Helpers

    The SDK provides helper methods for common streaming patterns:

    const runner = openai.beta.chat.completions.stream({
      model: "gpt-4o",
      messages: [{ role: "user", content: "Hello" }],
    });
    
    runner.on("content", (delta) => process.stdout.write(delta));
    const finalMessage = await runner.finalMessage();
    

    Key Takeaways

    • •Streaming reduces perceived latency by showing tokens as they generate
    • •Use SSE for web applications and async iterators for server-side
    • •Implement abort handling and error recovery for production streams
    • •SDK stream helpers simplify event handling and final message capture
    0 of 6 lessons marked complete

    Progress is kept in this browser only.