Skip to main content

​
Setup

Install the OpenRouter provider: 
pnpm add @openrouter/ai-sdk-provider
Create an OpenRouter provider instance: 
import { createOpenRouter } from '@openrouter/ai-sdk-provider';

const openrouter = createOpenRouter({
  apiKey: 'YOUR_OPENROUTER_API_KEY',
});

​
Using Models

// Chat models (recommended)
const chatModel = openrouter.chat('anthropic/claude-3.5-sonnet');

// Completion models
const completionModel = openrouter.completion('meta-llama/llama-3.1-405b-instruct');

​
Examples

​
Generate Text

import { createOpenRouter } from '@openrouter/ai-sdk-provider';
import { generateText } from 'ai';

const openrouter = createOpenRouter({
  apiKey: 'YOUR_OPENROUTER_API_KEY',
});

const { text } = await generateText({
  model: openrouter.chat('anthropic/claude-3.5-sonnet'),
  prompt: 'What is OpenRouter?',
});

​
Stream Text

import { createOpenRouter } from '@openrouter/ai-sdk-provider';
import { streamText } from 'ai';

const openrouter = createOpenRouter({
  apiKey: 'YOUR_OPENROUTER_API_KEY',
});

const result = streamText({
  model: openrouter.chat('meta-llama/llama-3.1-405b-instruct'),
  prompt: 'Write a short story about AI.',
});

for await (const chunk of result) {
  console.log(chunk);
}

​
Building AI Agents

AI agents are systems that can understand context and take meaningful actions. Let’s explore different approaches and patterns.

​
Building Blocks

  1. Single-Step LLM Generation
    • Basic building block for straightforward tasks
    • Used for classification or text generation
  2. Tool Usage
    • Enhanced capabilities through tools
    • Controlled way to extend LLM capabilities
  3. Multi-Agent Systems
    • Multiple LLMs working together
    • Specialized for different aspects of complex tasks

​
Workflow Patterns

  1. Sequential Processing
    • Steps executed in order
    • Each step’s output becomes input for the next
  2. Parallel Processing
    • Independent tasks run simultaneously
    • Improves efficiency for independent subtasks
  3. Evaluation/Feedback Loops
    • Results checked and improved iteratively
    • Ensures quality and accuracy
  4. Orchestration
    • Coordinating multiple components
    • Managing complex workflows
  5. Routing
    • Directing work based on context
    • Intelligent decision-making for workflow paths

​
Multi-Step Tool Usage

Use the maxSteps parameter to create agents that can solve complex problems through multiple steps: 
import { openai } from '@ai-sdk/openai';
import { generateText, tool } from 'ai';
import * as mathjs from 'mathjs';
import { z } from 'zod';

const { text: answer } = await generateText({
  model: openai('gpt-4o-2024-08-06', { structuredOutputs: true }),
  tools: {
    calculate: tool({
      description: 'A tool for evaluating mathematical expressions.',
      parameters: z.object({ expression: z.string() }),
      execute: async ({ expression }) => mathjs.evaluate(expression),
    }),
  },
  maxSteps: 10,
  system: 'You are solving math problems. Reason step by step.',
  prompt: 'Solve this math problem...',
});

​
Structured Answers

Use the answer tool to get structured outputs: 
const { toolCalls } = await generateText({
  model: openai('gpt-4o-2024-08-06', { structuredOutputs: true }),
  tools: {
    calculate: tool({
      description: 'A tool for evaluating mathematical expressions.',
      parameters: z.object({ expression: z.string() }),
      execute: async ({ expression }) => mathjs.evaluate(expression),
    }),
    answer: tool({
      description: 'A tool for providing the final answer.',
      parameters: z.object({
        steps: z.array(
          z.object({
            calculation: z.string(),
            reasoning: z.string(),
          }),
        ),
        answer: z.string(),
      }),
    }),
  },
  toolChoice: 'required',
  maxSteps: 10,
});

​
Monitoring Steps

Access information from all steps: 
const { steps } = await generateText({
  model: openai('gpt-4o'),
  maxSteps: 10,
});

// Get notified on each completed step
const result = await generateText({
  model: yourModel,
  maxSteps: 10,
  onStepFinish({ text, toolCalls, toolResults, finishReason, usage }) {
    // Handle step completion
  },
});

​
Best Practices

  1. Start Simple
    • Begin with basic implementations
    • Add complexity only when needed
  2. Error Handling
    • Implement retry mechanisms
    • Add fallback options
    • Log errors for debugging
  3. Performance
    • Cache frequently used results
    • Implement rate limiting
    • Use streaming for long operations
  4. Security
    • Validate tool inputs
    • Implement access control
    • Monitor tool usage
  5. Monitoring
    • Track agent performance
    • Log important decisions
    • Monitor resource usage