World of LLMs - Ollama + LangchainGo - Manual API Calls and Streaming Responses

In this blog post, we'll explore how to work with LLMs using Ollama and LangchainGo. We'll start by manually calling the Ollama API using curl locally and handling streaming responses. Then, we'll move on to using LangchainGo in Golang to communicate with the Ollama API/model.

Part 1: Manually Calling the Ollama API with Streaming Responses

Setting Up the Environment

First, ensure you have Ollama installed on your machine. Ollama allows you to run large language models (LLMs) locally, providing tools and APIs to interact with these models without sending data to third-party services. You can download and install Ollama from the official Ollama website.

Installing Llama 3 Locally

Before proceeding, make sure you have Llama 3 installed locally. You can do this by running the following command:

ollama pull llama3

This command downloads the Llama 3 model, making it available for local use.

Making a Manual API Call

To manually call the Ollama API using curl, follow these steps:

1. Start the Ollama server: Ensure the Ollama server is running. You can start it using the following command:

   ollama serve
   

2. Send a request using curl: Use curl to send a request to the Ollama API. Here’s an example command to send a prompt to the model and receive a response:

   curl http://localhost:11434/api/generate -d '{
        "model": "llama3",
        "prompt":"Why is the sky blue?"
    }'
   

Handling Streaming Responses

To handle streaming responses, we can add the "stream" : true to keep the connection open and receive data as it is generated by the model:

    curl http://localhost:11434/api/generate -d '{
        "model": "llama3",
        "prompt":"Why is the sky blue?",
        "stream": true
    }'

Part 2: Using LangchainGo to Communicate with Ollama API/Model

LangchainGo is a Go library for building applications with LLMs, providing a seamless way to interact with the Ollama API.

Setting Up LangchainGo

First, install LangchainGo in your Go project:

go get github.com/tmc/langchaingo

Using LangchainGo to Call the Ollama API

Here’s an example of how to use LangchainGo in a Go application to communicate with the Ollama API:

1. Create a new Go file (e.g., main.go):

package main

import (
    "context"
    "fmt"
    "github.com/tmc/langchaingo/llms/ollama"
)

func main() {
    client := ollama.NewClient("http://localhost:11434")

    prompt := "What is the capital of France?"
    response, err := client.Generate(context.Background(), "llama3", prompt)
    if err != nil {
        fmt.Println("Error generating response:", err)
        return
    }

    fmt.Println("Response:", response)
}

2. Run the Go application: Ensure the Ollama server is running, then run your Go application:

go run main.go

Handling Streaming Responses with LangchainGo

For streaming responses, you can use the WithStreamingFunc option in LangchainGo:

package main

import (
    "context"
    "fmt"
    "log"

    "github.com/tmc/langchaingo/llms"
    "github.com/tmc/langchaingo/llms/ollama"
)

func main() {
    client := ollama.NewClient("http://localhost:11434")

    prompt := "Tell me about the solar system"

    ctx := context.Background()
    _, err := llms.GenerateFromSinglePrompt(ctx, client, prompt,
        llms.WithStreamingFunc(func(ctx context.Context, chunk []byte) error {
            fmt.Printf("chunk: %s
", chunk)
            return nil
        }))
    if err != nil {
        log.Fatal(err)
    }
}

Conclusion

Using Ollama with LangchainGo provides a powerful way to interact with LLMs locally, offering flexibility and control over your data. By manually calling the Ollama API and handling streaming responses, you can build robust applications that leverage the capabilities of large language models. Integrating LangchainGo in your Go projects further simplifies the process, allowing seamless communication with the Ollama API/model.

For more information and detailed documentation, visit the Ollama website and the LangchainGo GitHub repository. Happy coding!