Skip to main content

Groq

Overview​

Groq serves open models (Llama, Mixtral, and others) over an OpenAI-compatible API with very low latency. Because the API is OpenAI-compatible, LangChainGo talks to Groq through the llms/openai package pointed at Groq's base URL — no separate provider package is required.

Prerequisites​

  • Go installed on your machine (version 1.24 or higher recommended).
  • A valid Groq API key. Obtain it by creating an account on the Groq platform and generating a new token.

Installation​

go get github.com/vxcontrol/langchaingo

Set your Groq API key as an environment variable:

export GROQ_API_KEY=your-api-key

Configuration​

The example builds an OpenAI client aimed at Groq's endpoint:

llm, err := openai.New(
    openai.WithModel("moonshotai/kimi-k2-instruct"),
    openai.WithBaseURL("https://api.groq.com/openai/v1"),
    openai.WithToken(os.Getenv("GROQ_API_KEY")),
)

The same pattern works for other OpenAI-compatible providers (DeepSeek, OpenRouter, NVIDIA, Ollama Cloud, …) — only the base URL, token, and model name change.

Usage​

package main

import (
	"context"
	"fmt"
	"log"
	"os"

	"github.com/vxcontrol/langchaingo/llms"
	"github.com/vxcontrol/langchaingo/llms/openai"
	"github.com/vxcontrol/langchaingo/llms/streaming"
)

func main() {
	apiKey := os.Getenv("GROQ_API_KEY")

	llm, err := openai.New(
		openai.WithModel("moonshotai/kimi-k2-instruct"),
		openai.WithBaseURL("https://api.groq.com/openai/v1"),
		openai.WithToken(apiKey),
	)
	if err != nil {
		log.Fatal(err)
	}
	ctx := context.Background()
	_, err = llms.GenerateFromSinglePrompt(ctx,
		llm,
		"Write a long poem about how golang is a fantastic language.",
		llms.WithTemperature(0.8),
		llms.WithMaxTokens(4096),
		llms.WithStreamingFunc(func(_ context.Context, chunk streaming.Chunk) error {
			fmt.Println(chunk.String())
			return nil
		}),
	)
	fmt.Println()
	if err != nil {
		log.Fatal(err)
	}
}