Skip to content

ai

Directory actions

More options

Directory actions

More options

Latest commit

 

History

History

ai

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
anthropic
anthropic
 
 
atlascloud
atlascloud
 
 
gemini
gemini
 
 
groq
groq
 
 
mistral
mistral
 
 
openai
openai
 
 
together
together
 
 
tools
tools
 
 
README.md
README.md
 
 
model.go
model.go
 
 
options.go
options.go
 
 

README.md

AI Package

The ai package provides a simple, high-level interface for AI model providers like Anthropic Claude and OpenAI GPT.

Interface

The Model interface follows the same patterns as other go-micro packages (Registry, Client, Broker):

type Model interface {
    Init(...Option) error
    Options() Options
    Generate(ctx context.Context, req *Request, opts ...GenerateOption) (*Response, error)
    Stream(ctx context.Context, req *Request, opts ...GenerateOption) (Stream, error)
    String() string
}

Quick Start

import (
    "context"
    "go-micro.dev/v5/ai"
    _ "go-micro.dev/v5/ai/anthropic"
    _ "go-micro.dev/v5/ai/openai"
)

// Create a model
m := ai.New("openai",
    ai.WithAPIKey("your-api-key"),
    ai.WithModel("gpt-4o"),
)

// Generate a response
req := &ai.Request{
    Prompt:       "What is Go?",
    SystemPrompt: "You are a helpful programming assistant",
}

resp, err := m.Generate(context.Background(), req)
if err != nil {
    log.Fatal(err)
}

fmt.Println(resp.Reply)

Options

Configure the model using functional options:

m := ai.New("anthropic",
    ai.WithAPIKey("your-key"),              // Required
    ai.WithModel("claude-sonnet-4-20250514"), // Optional, uses provider default
    ai.WithBaseURL("https://api.anthropic.com"), // Optional, uses provider default
)

You can also update options after creation:

m.Init(
    ai.WithModel("gpt-4o-mini"),
    ai.WithAPIKey("new-key"),
)

Using Tools

The model can automatically execute tool calls when provided with a tool handler:

// Define a tool handler
toolHandler := func(name string, input map[string]any) (result any, content string) {
    // Execute the tool and return results
    switch name {
    case "get_weather":
        return map[string]string{"temp": "72F"}, `{"temp": "72F"}`
    default:
        return nil, `{"error": "unknown tool"}`
    }
}

// Create model with tool handler
m := ai.New("openai",
    ai.WithAPIKey("your-key"),
    ai.WithToolHandler(toolHandler),
)

// Provide tools in the request
req := &ai.Request{
    Prompt: "What's the weather?",
    SystemPrompt: "You are a helpful assistant",
    Tools: []ai.Tool{
        {
            Name:        "get_weather",
            Description: "Get current weather",
            Properties: map[string]any{
                "location": map[string]any{
                    "type": "string",
                    "description": "City name",
                },
            },
        },
    },
}

// Generate will automatically call tools and return final answer
resp, err := m.Generate(context.Background(), req)
fmt.Println(resp.Answer) // Final answer after tool execution

Response Structure

type Response struct {
    Reply     string      // Initial reply from model
    ToolCalls []ToolCall  // Tools the model wants to call
    Answer    string      // Final answer (after tool execution if handler provided)
}
  • Reply: The model's first response
  • ToolCalls: List of tools the model requested (if any)
  • Answer: The final answer after tools are executed (only set if ToolHandler is provided)

Supported Providers

Anthropic Claude

m := ai.New("anthropic",
    ai.WithAPIKey("sk-ant-..."),
    ai.WithModel("claude-sonnet-4-20250514"), // default
)

Default model: claude-sonnet-4-20250514 Default base URL: https://api.anthropic.com

OpenAI GPT

m := ai.New("openai",
    ai.WithAPIKey("sk-..."),
    ai.WithModel("gpt-4o"), // default
)

Default model: gpt-4o Default base URL: https://api.openai.com

Google Gemini

m := ai.New("gemini",
    ai.WithAPIKey("your-key"),
    ai.WithModel("gemini-2.5-flash"), // default
)

Default model: gemini-2.5-flash Default base URL: https://generativelanguage.googleapis.com

Google Gemini uses its own API format with system_instruction, contents (not messages), and functionDeclarations for tool calling. The provider handles the translation automatically.

Groq

m := ai.New("groq",
    ai.WithAPIKey("your-key"),
    ai.WithModel("llama-3.3-70b-versatile"), // default
)

Default model: llama-3.3-70b-versatile Default base URL: https://api.groq.com/openai

Groq provides ultra-fast inference for open-weight models via an OpenAI-compatible endpoint.

Mistral

m := ai.New("mistral",
    ai.WithAPIKey("your-key"),
    ai.WithModel("mistral-large-latest"), // default
)

Default model: mistral-large-latest Default base URL: https://api.mistral.ai

Mistral AI is a European AI company offering high-performance models via an OpenAI-compatible endpoint.

Together AI

m := ai.New("together",
    ai.WithAPIKey("your-key"),
    ai.WithModel("meta-llama/Llama-3.3-70B-Instruct-Turbo"), // default
)

Default model: meta-llama/Llama-3.3-70B-Instruct-Turbo Default base URL: https://api.together.xyz

Together AI provides fast inference for open-weight models via an OpenAI-compatible endpoint.

Atlas Cloud

m := ai.New("atlascloud",
    ai.WithAPIKey("your-key"),
    ai.WithModel("llama-3.3-70b"), // default
)

Default model: llama-3.3-70b Default base URL: https://api.atlascloud.ai

Atlas Cloud is an enterprise AI infrastructure platform offering high-performance LLM APIs. It exposes an OpenAI-compatible chat completions endpoint with tool calling support.

Auto-Detection

Use AutoDetectProvider() to detect the provider from a base URL:

provider := ai.AutoDetectProvider("https://api.anthropic.com")
// Returns "anthropic"

m := ai.New(provider, ai.WithAPIKey("..."))

Adding a New Provider

See the full AI Provider Integration Guide for a step-by-step walkthrough, checklist, and design notes.

Quick summary:

  1. Create ai/yourprovider/yourprovider.go implementing ai.Model.
  2. Call ai.Register("yourprovider", ...) in init().
  3. Add tests in ai/yourprovider/yourprovider_test.go.
  4. Users enable the provider with a blank import:
import _ "go-micro.dev/v5/ai/yourprovider"

We welcome contributions and sponsorships from AI infrastructure companies — see the guide for details.

Comparison with Other Packages

The ai package follows the same patterns as other go-micro packages:

Registry:

r := registry.NewRegistry(registry.Addrs("..."))
r.Register(service)

Client:

c := client.NewClient(client.Retries(3))
c.Call(ctx, req, rsp)

AI:

m := ai.New("openai", ai.WithAPIKey("..."))
m.Generate(ctx, req)

All use:

  • Init() to update options
  • Options() to get current options
  • String() to get the implementation name
  • Functional options pattern

Testing

go test ./ai/...

Examples

See the server implementation for a complete example of using the ai package with tool execution.