AI integration

Use OpenAI, Claude, and Grok in your application with streaming responses

This boilerplate includes integrations for multiple AI providers with support for streaming responses, making it easy to build AI-powered features.

Overview

The AI integration provides:

Multiple providers - OpenAI (GPT-4), Anthropic (Claude), and Grok (xAI)
Streaming responses - Real-time token-by-token output
Type-safe API - Zod validation for requests
Server-side processing - Secure API key management
Flexible configuration - Easy to add more providers

Configuration

Environment variables

Add your AI provider API keys to .env:

.env

# OpenAI (for GPT models)
OPENAI_API_KEY="sk-..."

# Anthropic (for Claude models)
ANTHROPIC_API_KEY="sk-ant-..."

# Grok / xAI (for Grok models)
GROK_API_KEY="xai-..."

You only need to configure the providers you plan to use. Get API keys from:

API endpoint

The streaming API endpoint is located in server/api/ai/stream.ts:

server/api/ai/stream.ts

import { OpenAI } from 'openai'
import Anthropic from '@anthropic-ai/sdk'
import { z } from 'zod'

const StreamRequestSchema = z.object({
  model: z.enum(['chatgpt', 'claude', 'grok']),
  prompt: z.string().min(1).max(4000),
  temperature: z.number().min(0).max(2).default(0.3),
  max_tokens: z.number().int().min(1).max(4000).default(2000),
  top_p: z.number().min(0).max(1).default(0.95),
  stream: z.boolean().default(true),
})

export default defineEventHandler(async event => {
  const body = await readBody(event)
  const { model, prompt, temperature, max_tokens, top_p } = StreamRequestSchema.parse(body)

  // Handle streaming based on provider
  // ... (see implementation in the file)
})

Supported models

The boilerplate is configured with these models by default:

const models = {
  chatgpt: 'gpt-4o-mini', // Fast, cost-effective GPT-4
  claude: 'claude-3-5-haiku-latest', // Fast Claude model
  grok: 'grok-4', // xAI's Grok model
}

You can easily change these to other models:

const models = {
  chatgpt: 'gpt-4o', // More capable GPT-4
  claude: 'claude-3-5-sonnet-latest', // More capable Claude
  grok: 'grok-vision-beta', // Grok with vision
}

Using the AI API

Client-side example

Here's how to use the streaming API in your components:

<script setup lang="ts">
const prompt = ref('')
const response = ref('')
const isStreaming = ref(false)
const selectedModel = ref<'chatgpt' | 'claude' | 'grok'>('chatgpt')

async function handleSubmit() {
  if (!prompt.value.trim()) return

  isStreaming.value = true
  response.value = ''

  try {
    const res = await fetch('/api/ai/stream', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({
        model: selectedModel.value,
        prompt: prompt.value,
        temperature: 0.7,
        max_tokens: 2000,
      }),
    })

    if (!res.ok) throw new Error('Failed to get response')

    const reader = res.body?.getReader()
    const decoder = new TextDecoder()

    while (true) {
      const { done, value } = await reader!.read()
      if (done) break

      const chunk = decoder.decode(value, { stream: true })
      response.value += chunk
    }
  } catch (error) {
    console.error('AI error:', error)
    toast.error('Failed to get AI response')
  } finally {
    isStreaming.value = false
  }
}
</script>

<template>
  <div class="space-y-4">
    <div class="space-y-2">
      <Label>Select AI Model</Label>
      <Select v-model="selectedModel">
        <SelectTrigger>
          <SelectValue />
        </SelectTrigger>
        <SelectContent>
          <SelectItem value="chatgpt">ChatGPT (GPT-4)</SelectItem>
          <SelectItem value="claude">Claude (Anthropic)</SelectItem>
          <SelectItem value="grok">Grok (xAI)</SelectItem>
        </SelectContent>
      </Select>
    </div>

    <div class="space-y-2">
      <Label for="prompt">Your prompt</Label>
      <Textarea id="prompt" v-model="prompt" placeholder="Ask me anything..." rows="4" />
    </div>

    <Button @click="handleSubmit" :disabled="isStreaming">
      <span v-if="isStreaming">
        <Icon name="lucide:loader-2" class="animate-spin mr-2" />
        Generating...
      </span>
      <span v-else>Send</span>
    </Button>

    <Card v-if="response" class="mt-4">
      <CardHeader>
        <CardTitle>Response</CardTitle>
      </CardHeader>
      <CardContent>
        <div class="prose dark:prose-invert max-w-none">
          {{ response }}
        </div>
      </CardContent>
    </Card>
  </div>
</template>

Composable for AI streaming

Create a reusable composable for AI interactions:

app/composables/useAIStream.ts

export function useAIStream() {
  const response = ref('')
  const isStreaming = ref(false)
  const error = ref<string | null>(null)

  async function stream(params: {
    model: 'chatgpt' | 'claude' | 'grok'
    prompt: string
    temperature?: number
    max_tokens?: number
  }) {
    isStreaming.value = true
    response.value = ''
    error.value = null

    try {
      const res = await fetch('/api/ai/stream', {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({
          model: params.model,
          prompt: params.prompt,
          temperature: params.temperature ?? 0.7,
          max_tokens: params.max_tokens ?? 2000,
        }),
      })

      if (!res.ok) {
        throw new Error('Failed to get AI response')
      }

      const reader = res.body?.getReader()
      const decoder = new TextDecoder()

      while (true) {
        const { done, value } = await reader!.read()
        if (done) break

        const chunk = decoder.decode(value, { stream: true })
        response.value += chunk
      }
    } catch (err) {
      error.value = err instanceof Error ? err.message : 'Unknown error'
    } finally {
      isStreaming.value = false
    }
  }

  function reset() {
    response.value = ''
    error.value = null
  }

  return {
    response: readonly(response),
    isStreaming: readonly(isStreaming),
    error: readonly(error),
    stream,
    reset,
  }
}

Then use it in your components:

<script setup lang="ts">
const { response, isStreaming, error, stream, reset } = useAIStream()

async function handleSubmit() {
  await stream({
    model: 'chatgpt',
    prompt: 'Explain quantum computing in simple terms',
    temperature: 0.7,
  })
}
</script>

Protecting AI endpoints

Require authentication

Only allow authenticated users to access AI features:

server/api/ai/stream.ts

import { requireAuth } from '~/server/utils/require-auth'

export default defineEventHandler(async event => {
  // Require authentication
  const { user } = await requireAuth(event)

  // ... rest of the code
})

Require subscription

Only allow paying subscribers to use AI:

server/api/ai/stream.ts

import { requireSubscription } from '~/server/utils/require-subscription'

export default defineEventHandler(async event => {
  // Require pro or elite subscription
  await requireSubscription(event, { plans: ['pro', 'elite'] })

  // ... rest of the code
})

Rate limiting

Implement rate limiting to prevent abuse:

server/api/ai/stream.ts

import { createRateLimiter } from '~/server/utils/rate-limit'

const limiter = createRateLimiter({
  interval: 60000, // 1 minute
  max: 10, // 10 requests per minute
})

export default defineEventHandler(async event => {
  await limiter.check(event)

  // ... rest of the code
})

Model configurations

Temperature

Controls randomness (0-2):

0.0-0.3: Focused, deterministic (good for facts, code)
0.7-1.0: Balanced creativity
1.5-2.0: Very creative, unpredictable

await stream({
  model: 'chatgpt',
  prompt: 'Write a poem',
  temperature: 1.2, // More creative
})

Max tokens

Controls response length:

GPT-4: Up to 128,000 tokens context
Claude: Up to 200,000 tokens context
Grok: Check current limits

await stream({
  model: 'claude',
  prompt: 'Explain machine learning',
  max_tokens: 500, // Shorter response
})

Top P

Alternative to temperature for controlling randomness (0-1):

await stream({
  model: 'chatgpt',
  prompt: 'Generate ideas',
  top_p: 0.9,
})

Adding more providers

To add a new AI provider:

Install the SDK:

pnpm add @google/generative-ai

Add to the stream handler:

server/api/ai/stream.ts

import { GoogleGenerativeAI } from '@google/generative-ai'

const models = {
  // ... existing models
  gemini: 'gemini-1.5-flash',
}

const gemini = new GoogleGenerativeAI({ apiKey: process.env.GEMINI_API_KEY })

// In the handler
case 'gemini': {
  const model = gemini.getGenerativeModel({ model: models.gemini })
  const result = await model.generateContentStream(prompt)

  for await (const chunk of result.stream) {
    const text = chunk.text()
    if (text) controller.enqueue(encoder.encode(text))
  }
  break
}

Update the schema:

const StreamRequestSchema = z.object({
  model: z.enum(['chatgpt', 'claude', 'grok', 'gemini']),
  // ...
})

Use cases

AI chat assistant

Build a chatbot that maintains conversation history:

<script setup lang="ts">
const messages = ref<Array<{ role: 'user' | 'assistant'; content: string }>>([])
const { stream, isStreaming, response } = useAIStream()

async function sendMessage(content: string) {
  messages.value.push({ role: 'user', content })

  const prompt = messages.value.map(m => `${m.role}: ${m.content}`).join('\n') + '\nassistant:'

  await stream({ model: 'chatgpt', prompt })
  messages.value.push({ role: 'assistant', content: response.value })
}
</script>

Content generation

Generate blog posts, emails, or marketing copy:

await stream({
  model: 'claude',
  prompt: `Write a blog post about ${topic}. Include:
    - An engaging introduction
    - 3 main points with examples
    - A conclusion with a call to action
    Target audience: ${audience}`,
  temperature: 0.8,
  max_tokens: 2000,
})

Code assistance

Help users with coding questions:

await stream({
  model: 'chatgpt',
  prompt: `Debug this code and explain the issue:
    
    \`\`\`javascript
    ${userCode}
    \`\`\`
    
    Error: ${errorMessage}`,
  temperature: 0.3, // Lower for more precise code help
})

Data analysis

Analyze and summarize data:

await stream({
  model: 'grok',
  prompt: `Analyze this sales data and provide insights:
    ${JSON.stringify(salesData)}
    
    Focus on:
    - Top performing products
    - Revenue trends
    - Recommendations`,
})

Best practices

Handle errors gracefully

try {
  await stream({ model: 'chatgpt', prompt: userInput })
} catch (error) {
  if (error.status === 429) {
    toast.error('Rate limit exceeded. Please try again later.')
  } else {
    toast.error('Failed to generate response. Please try again.')
  }
}

Sanitize user input

function sanitizePrompt(input: string): string {
  // Remove harmful content, limit length, etc.
  return input
    .trim()
    .substring(0, 4000) // Max length
    .replace(/[<>]/g, '') // Remove HTML
}

Monitor costs

AI APIs can be expensive. Monitor usage:

// Log token usage
const usage = response.usage
logger.info('AI request', {
  model,
  promptTokens: usage.prompt_tokens,
  completionTokens: usage.completion_tokens,
  totalCost: calculateCost(usage),
})

Cache responses

Cache common queries to reduce API calls:

const cacheKey = `ai:${model}:${hash(prompt)}`
const cached = await redis.get(cacheKey)

if (cached) {
  return cached
}

const response = await stream({ model, prompt })
await redis.set(cacheKey, response, 'EX', 3600) // Cache for 1 hour

Troubleshooting

API key errors

Verify your environment variables are set:

echo $OPENAI_API_KEY
echo $ANTHROPIC_API_KEY

Rate limit errors

Check your API usage in the provider dashboard
Implement request queuing
Upgrade your API plan

Timeout errors

Reduce max_tokens
Break large requests into smaller chunks
Increase server timeout settings

Model not available

Check model name spelling
Verify you have access to the model
Check provider documentation for available models

Reference

AI providers frequently update their models and pricing. Always check the official documentation for the latest information.

Payments

Accept payments and manage subscriptions with Stripe

User feedback

Collect bug reports, feature requests, and support messages