Agora Conversational AI Next.js Quickstart

Build a production-style voice agent in minutes with Next.js and the Agora Conversational AI Engine, including voice agent visualizer (Agent UIKit), live transcript, and real-time pipeline latency via AGENT_METRICS (Agent Toolkit).

Prerequisites

• Node.js 22+
• pnpm
• Agora CLI

Run It

Getting started is quick and easy: install the CLI (skip if you already have it) , scaffold the Next.js quickstart using the Agora CLI, install dependencies, and run.

1. Install the Agora CLI and sign in (skip if agora is already on your PATH):

   macOS and Linux:

   curl -fsSL https://raw.githubusercontent.com/AgoraIO/cli/main/install.sh | sh -s -- --add-to-path

   Windows PowerShell:

   irm https://dl.agora.io/cli/install.ps1 | iex

   If the Windows install command fails in PowerShell, try running the macOS/Linux command from Git Bash, then open a new terminal and run agora --help to confirm the CLI is on your PATH.

   Then verify and sign in:

   agora --help
   agora login

   If agora --help is not found after install, close and reopen your terminal, then try again. If it still fails, check that the installer-added Agora CLI location is on your shell PATH.

2. Scaffold and run agora init clones the starter, binds an Agora project, and writes .env.local. (replace my-nextjs-demo with your own project name):

   agora init my-nextjs-demo --template nextjs
   cd my-nextjs-demo
   pnpm install
   pnpm dev

3. Open http://localhost:3000 and click Start conversation.

If the agent does not join or transcripts do not appear, run agora project doctor --deep to check credentials, feature enablement, network reachability, and local env binding.

Working from a clone of this repository

Use this path if you already cloned this repo (for example to contribute or fork):

git clone https://github.com/AgoraIO-Conversational-AI/agent-quickstart-nextjs.git
cd agent-quickstart-nextjs
agora login
agora project use <your-project>
pnpm install
agora project env write .env.local
agora project doctor --deep
pnpm dev

Deploy to Vercel

To populate Vercel env vars from your bound Agora project:

agora project use <your-project>
agora project env write .env.local
rg "^(NEXT_PUBLIC_AGORA_APP_ID|NEXT_AGORA_APP_CERTIFICATE)=" .env.local

Copy those two values into Vercel Project Settings -> Environment Variables.

Environment variables

Defined in env.local.example.

Variable	Required	Notes
NEXT_PUBLIC_AGORA_APP_ID	✅	Agora Console → Project → App ID.
NEXT_AGORA_APP_CERTIFICATE	✅	Agora Console → Project → App Certificate. Server-side only.

The default agent configuration in app/api/invite-agent/route.ts uses Agora-managed STT, LLM, and TTS, so no extra vendor API keys are required for the base quickstart.

Commands

# Dev
pnpm dev                # start the Next.js dev server

# Quality
pnpm run lint           # eslint
pnpm run typecheck      # tsc --noEmit
pnpm run doctor         # local prereqs + env binding

# CI / pre-ship
pnpm run verify:api     # API contract checks
pnpm run build          # production build
pnpm run verify         # doctor + lint + typecheck + verify:api + build

Run pnpm run verify before shipping changes — it covers local prerequisites, lint, type safety, the core API route contracts, and the production build.

Architecture

The browser fetches a combined RTC + RTM token (buildTokenWithRtm) from this app, joins the channel using a single RTC client, and uses RTM as the data channel for transcript, agent state, metrics, and error events. The Conversational AI Engine joins the same channel as the shared agent UID in lib/agora.ts and runs the STT → LLM → TTS pipeline in Agora Cloud.

What You Get

• browser voice client built with Next.js App Router
• RTC audio plus RTM transcript and state events
• server routes for token generation, invite, and stop
• AgentVisualizer for agent state and a built-in transcript panel for live turns
• per-stage latency header driven by AGENT_METRICS
• Agora-managed default STT, LLM, and TTS configuration

How It Works

1. The browser requests an RTC + RTM token from /api/generate-agora-token.
2. The backend invites an Agora cloud agent with /api/invite-agent.
3. The browser joins the channel and publishes mic audio.
4. The client receives transcript, agent state, and AGENT_METRICS (per-stage latency) events over RTM.
5. On end, the client calls /api/stop-conversation, logs out RTM, and unmounts the call view so Agora React hooks clean up RTC publish/join and the local microphone track.

Optional BYOK

The base .env.local contract contains only Agora credentials. If you are migrating from a supported provider, uncomment the matching snippet in app/api/invite-agent/route.ts and add its variables to your local environment.

# Deepgram STT
NEXT_DEEPGRAM_API_KEY=...

# OpenAI-compatible LLM
NEXT_LLM_URL=https://api.openai.com/v1/chat/completions
NEXT_LLM_API_KEY=...

# ElevenLabs TTS
NEXT_ELEVENLABS_API_KEY=...
NEXT_ELEVENLABS_VOICE_ID=...

Repo Map

• app/api/generate-agora-token/route.ts — issues RTC + RTM tokens
• app/api/invite-agent/route.ts — starts the agent session and configures the pipeline
• app/api/stop-conversation/route.ts — stops the agent session
• components/LandingPage.tsx — entry point: token fetch, RTM login, conversation lifecycle
• components/ConversationComponent.tsx — RTC client, transcript state, AGENT_METRICS, mic release
• components/QuickstartConversationLayout.tsx — in-call header, transcript rail, controls dock
• components/QuickstartPipelineMetrics.tsx — per-stage latency chips in the header
• components/QuickstartTranscriptPanel.tsx — live transcript rail
• components/QuickstartPreCallCard.tsx — pre-call hero card
• lib/conversation.ts — transcript normalization and visualizer state mapping
• AGENTS.md — primary agent-facing guide

Troubleshooting

• Agent does not join or transcripts are missing: run agora project doctor --deep.
• pnpm run doctor fails: run agora project env write .env.local, then retry.
• Manual clone / env values: agora project use <your-project> then agora project env write .env.local.
• RTM login fails: keep app/api/generate-agora-token/route.ts on RtcTokenBuilder.buildTokenWithRtm — RTC-only tokens will not satisfy rtm.login.
• Transcript speakers inverted: check the uid === "0" remap in components/ConversationComponent.tsx.
• Agent never appears in channel: ensure the shared agent UID in lib/agora.ts is used by both the client and invite route.

More Docs

• docs/ai/L0_repo_card.md
• docs/ai/RECIPE.md
• AGENTS.md

Contributing

Pull requests welcome — see CONTRIBUTING.md for development setup and conventions.

Security

Please do not open public issues for security reports. Email security@agora.io with details and reproduction steps.

License

Released under the MIT License.