AI Model Wiki

    Skip to main content
    40 min
    Lesson 3 of 5

    Streaming Types & Patterns

    From the course TypeScript for AI Applications

    Type-Safe Streaming

    Typed Stream Events

    type StreamEvent =
      | { type: "text_delta"; content: string }
      | { type: "tool_call_start"; id: string; name: string }
      | { type: "tool_call_delta"; id: string; arguments: string }
      | { type: "done"; usage: { promptTokens: number; completionTokens: number } }
      | { type: "error"; message: string };
    
    async function* streamChat(messages: ChatMessage[]): AsyncGenerator<StreamEvent> {
      const stream = await openai.chat.completions.create({
        model: "gpt-4o",
        messages,
        stream: true,
      });
    
      for await (const chunk of stream) {
        const delta = chunk.choices[0]?.delta;
        if (delta?.content) {
          yield { type: "text_delta", content: delta.content };
        }
        if (delta?.tool_calls) {
          for (const tc of delta.tool_calls) {
            if (tc.function?.name) {
              yield { type: "tool_call_start", id: tc.id!, name: tc.function.name };
            }
            if (tc.function?.arguments) {
              yield { type: "tool_call_delta", id: tc.id!, arguments: tc.function.arguments };
            }
          }
        }
      }
      yield { type: "done", usage: { promptTokens: 0, completionTokens: 0 } };
    }
    

    React Hook for Streaming

    function useStreamChat() {
      const [messages, setMessages] = useState<ChatMessage[]>([]);
      const [isStreaming, setIsStreaming] = useState(false);
    
      const send = useCallback(async (content: string) => {
        setIsStreaming(true);
        setMessages(prev => [...prev, { role: "user", content }]);
    
        let assistantContent = "";
        for await (const event of streamChat([...messages, { role: "user", content }])) {
          if (event.type === "text_delta") {
            assistantContent += event.content;
            setMessages(prev => {
              const updated = [...prev];
              updated[updated.length - 1] = { role: "assistant", content: assistantContent };
              return updated;
            });
          }
        }
        setIsStreaming(false);
      }, [messages]);
    
      return { messages, send, isStreaming };
    }
    

    Common Pitfall: Not Handling the done Event in Streaming Consumers

    An AsyncGenerator terminates when it returns. If you only handle text_delta events and skip done, you miss the chance to run cleanup code (saving the final message, clearing loading state):

    for await (const event of streamChat(messages)) {
      if (event.type === "text_delta") {
        content += event.content;
      } else if (event.type === "done") {
        await db.saveMessage({ content, usage: event.usage });
        setIsStreaming(false);
      } else if (event.type === "error") {
        setError(event.message);
        break;
      }
    }
    

    Best Practice: Limit Parallel Tool Call Concurrency

    When the model requests parallel tool calls, cap simultaneous execution to avoid overwhelming external APIs:

    import pLimit from "p-limit";
    const limit = pLimit(3);
    
    const results = await Promise.all(
      toolCalls.map(call => limit(() => executeFunction(call.name, call.arguments)))
    );
    

    Named Product Reference: Vercel AI SDK generateObject

    The Vercel AI SDK's generateObject function combines Zod schema validation with structured output in one call. Pass your Zod schema as the schema parameter and receive a fully typed object with compile-time and runtime guarantees. It automatically handles JSON Schema generation and response parsing, eliminating the zodToJsonSchema conversion step.

    Key Takeaways

    • •Use discriminated unions for typed stream events
    • •AsyncGenerator provides a clean streaming interface
    • •Build React hooks that encapsulate streaming logic
    • •Handle text deltas, tool calls, and errors with exhaustive types
    0 of 5 lessons marked complete

    Progress is kept in this browser only.