Adding a chat box backed by a large language model is one of those tasks that looks trivial in a demo and turns into a week of edge cases in production. The demo streams tokens beautifully; then you ship it and discover you have no rate limiting, no way to cancel a runaway response, and a bill that scales with whoever feels like spamming your endpoint.
The Vercel AI SDK smooths over the boring parts — streaming, message state, provider differences — so you can spend your time on the parts that matter. This post walks through a minimal-but-honest setup in the Next.js App Router, then spends most of its words on the details you only learn by shipping.
Note: the AI SDK moves fast, and import paths shift between major versions. Treat the snippets below as the shape of the solution and confirm the exact imports against the official docs for the version you install.
Why reach for the SDK at all#
You can call any model's HTTP API yourself. For a one-off script, do that. The moment you want streaming into a React UI, you're suddenly hand-rolling Server-Sent Events parsing, an incremental message buffer, abort handling, and a state machine for "thinking / streaming / done / errored". That is exactly the undifferentiated plumbing the SDK owns:
- Provider abstraction — swap between OpenAI, Anthropic, Google, or a local model by changing one line, because the message format is normalized.
- Streaming primitives — a server helper that returns a streaming
Response, and a client hook that consumes it. - UI state — the running list of messages, the in-flight input, and loading flags, without you writing another reducer.
The server: a streaming route handler#
In the App Router, a chat endpoint is a normal POST route handler. It reads the message history the client sends, hands it to the model, and returns a streaming response:
// app/api/chat/route.ts
import { anthropic } from '@ai-sdk/anthropic';
import { streamText } from 'ai';
// Let responses stream for up to 30s instead of the default short budget.
export const maxDuration = 30;
export async function POST(req: Request) {
const { messages } = await req.json();
const result = streamText({
// Pick a current model id from your provider's model list.
model: anthropic('claude-sonnet-4-5'),
system: 'You are a concise, friendly assistant for a developer portfolio.',
messages,
});
return result.toDataStreamResponse();
}
Two things worth internalizing here. First, the route is stateless — the client owns the conversation and replays it on every turn. That keeps your server simple but means the full history counts as input tokens every request, which matters for cost (more on that below). Second, streamText returns immediately; the actual model call happens as the response streams, so you are not blocking the event loop waiting for a full completion.
The client: the useChat hook#
On the client, one hook wires up the whole interaction — message list, input binding, and submission:
'use client';
import { useChat } from '@ai-sdk/react';
export default function Chat() {
const { messages, input, handleInputChange, handleSubmit, status } =
useChat();
return (
<div>
<ul>
{messages.map(message => (
<li key={message.id}>
<strong>{message.role}: </strong>
{message.content}
</li>
))}
</ul>
<form onSubmit={handleSubmit}>
<input
value={input}
onChange={handleInputChange}
placeholder="Ask something…"
disabled={status !== 'ready'}
/>
<button type="submit">Send</button>
</form>
</div>
);
}
That is a working chat UI. Tokens appear as they arrive because the hook appends streamed deltas to the last assistant message for you. Everything past this point is the difference between a demo and a feature.
The details the tutorials skip#
Rate limiting is not optional. An unauthenticated, unthrottled model endpoint is a way to convert your API budget into someone else's entertainment. Put a limiter in front of the route — keyed by IP for anonymous traffic, by user id once you have auth. A fixed window (say, N requests per minute) is enough to start; reach for a sliding window only if abuse patterns demand it.
Let users cancel. Long generations are expensive and sometimes go sideways. The hook exposes a stop() function; wire it to a button that appears while status === 'streaming'. Stopping aborts the underlying request, so you stop paying for tokens you'll never show.
Cap the history you send. Because the conversation is replayed every turn, a long chat quietly inflates input tokens on each message. Trim to a rolling window of recent turns, or summarize older ones, before they reach the model. Your future self reading the invoice will thank you.
Handle the error path visibly. Networks blip, providers rate-limit you, and models occasionally refuse. Surface a real error state in the UI instead of a silently stuck spinner — the hook gives you an error value and a reload() to retry the last turn.
Never trust the output as safe HTML. If you render model output as Markdown, sanitize it the same way you would any user-generated content. The model can be coaxed into emitting things you would not want injected into your page.
Where to go next#
Once the basics are solid, the interesting work begins: giving the model tools so it can look things up or take actions, grounding it in your own content with retrieval, and persisting conversations so users can return to them. Tools and retrieval especially are where a chat box turns from a novelty into something genuinely useful — but they only pay off once the streaming, limiting, and error handling underneath are boring and reliable.
Start small, ship the plumbing first, and add capability once the foundation stops surprising you.
FAQ#
Do I need the AI SDK to build AI chat in Next.js?#
No — you can call any model's HTTP API directly from a route handler. The SDK earns its place once you want token streaming into a React UI, since it handles the Server-Sent Events parsing, message state, and abort logic you would otherwise write by hand.
Should the chat route run on the Edge or Node.js runtime?#
Either works. The Edge runtime gives lower cold-start latency and is a natural fit for streaming, but has a more limited API surface. If your route needs Node-only libraries (some database drivers, certain SDKs), stay on the Node.js runtime. Start with the default and switch only if you measure a reason to.
How do I stop the response from running up my API bill?#
Three levers: rate-limit the endpoint so it can't be spammed, expose a stop button so users can cancel long generations, and trim the conversation history you replay each turn so input tokens don't grow without bound.
Can I switch model providers later?#
Yes — that's a core reason to use the SDK. Because it normalizes the message format across providers, swapping from one provider to another is usually a matter of changing the model import and id, not rewriting your route or UI.
References: