Have you ever wondered how AI accurately finds the answer when you ask "What is this year's annual leave policy?" in your company's knowledge base?

Behind this question lies a complete workflow. Today we will discuss it in plain English.

I. Why choose Go instead of Python?

Let's first answer a question that everyone asks: Python is clearly the most popular language in the AI field, so why use Go?

1.1 An analogy makes it clear

If writing code is like cooking:

• Python is like a fully equipped large kitchen, suitable for researching new dishes (training models, doing experiments).
• Go is like the central kitchen of a fast-food chain, suitable for stable, high-volume, and fast delivery of dishes (handling user requests).

A RAG (Retrieval-Augmented Generation) knowledge base system essentially belongs to the latter. Its workflow is: Receive user query → Call AI API → Wait for result → Return to user. During this process, 90% of the time is spent waiting for network responses, not doing math calculations. Go happens to excel at handling this "waiting" phase.

1.2 Concurrency capability: Go has countless helpers

Imagine queuing up at a bubble tea shop:

• Python is like having one shop assistant who can only serve one customer at a time (due to the Global Interpreter Lock, or GIL). If 100 customers arrive, the other 99 have to wait.
• Go is like having thousands of shop assistants; every customer can be served immediately. Moreover, these "assistants" (Goroutines) consume very little memory, allowing a standard server to handle tens of thousands of requests concurrently.

1.3 Deployment: One file does it all

When deploying Python projects, you often encounter situations where it runs on your local machine but fails on the server due to environment incompatibilities or network timeouts during dependency installations. In contrast, Go compiles into a single binary file. You can simply upload it to the server and run it—offering a deployment experience as worry-free as "moving in with just your bags."

1.4 Go's weaknesses happen to be irrelevant here

Some say Go's AI ecosystem is inferior to Python's—and they are right. But RAG systems do not require training models locally; they simply call AI APIs. What you need is connection management, database read/writes, and task scheduling, which are precisely Go's strengths. AI is responsible for "thinking" and Go is responsible for "organizing"; they perform their respective duties.

II. The first problem: What if there are all kinds of file formats?

Files in a corporate knowledge base are extremely diverse: Word, PDF, TXT, Markdown, HTML, and so on. The first step of vectorization is to convert these files into plain text.

2.1 A lazy yet smart trick

If we write parser code for each format, the maintenance cost will be very high. Our approach is to find a "translator." We upload the file to Google Drive, use its built-in conversion feature to convert it to Google Docs format, and then export it as plain text.

// Check file type: only proceed to AI processing if it can be converted to text
switch ext {
case ".pdf", ".docx", ".doc", ".txt", ".html", ".md":
    // Automatically convert to Google Docs format during upload
    // Cloud storage will handle the format conversion for us
    canConvert = true
}

// Extract text: regardless of whether the original file is PDF or Word, export it unified as plain text
func ExtractText(fileID string) (string, error) {
    resp, err := service.Files.Export(fileID, "text/plain").Download()
    if err != nil {
        return "", err
    }
    defer resp.Body.Close()
    
    content, err := io.ReadAll(resp.Body)
    if err != nil {
        return "", err
    }
    return string(content), nil
}

The benefit is straightforward: our code only needs to handle "plain text", leaving all format compatibility issues to cloud services like Google.

2.2 Deduplication along the way

We calculate a "fingerprint"—known as the SHA256 hash—for the file during upload. If two people upload the same file, the fingerprint will be identical, allowing us to skip processing directly, saving time and effort.

III. The second problem: How to chunk long documents?

AI has a "reading limit" (context window limit). You cannot throw the entire Three-Body Problem novel to it at once; you must cut it into small chunks.

3.1 Why can't we just make a clean cut?

Imagine this sentence:

"According to company regulations, employees are entitled to 15 days of paid annual leave each year, which can be used in installments."

If we happen to cut it right between "employees" and "are entitled":

• First chunk: "According to company regulations, employees"
• Second chunk: "are entitled to 15 days of paid annual leave each year, which can be used in installments."

Looking at either chunk alone makes the semantics very weird. The solution is overlapping chunking—letting the chunks overlap with each other:

• First chunk: [According to company regulations, employees are entitled to 15 days]
• Second chunk: [are entitled to 15 days of paid annual leave each year, which can be used in installments]

See? "are entitled to 15 days" appears in both chunks. This way, no matter which chunk is retrieved, key semantic information is not lost. The code implementation is also straightforward:

// ChunkText overlapping chunking algorithm
// chunkSize = word count per chunk, overlap = overlapping word count
func ChunkText(text string, chunkSize int, overlap int) []string {
    runes := []rune(text) // Process Chinese correctly by slicing by characters
    var chunks []string
    
    for i := 0; i < len(runes); {
        end := i + chunkSize
        if end > len(runes) {
            end = len(runes)
        }
        chunks = append(chunks, string(runes[i:end]))
        if end == len(runes) {
            break
        }
        i = end - overlap // Go back overlap characters to implement overlap
    }
    return chunks
}

[!IMPORTANT]
Make sure to use []rune instead of []byte—Chinese characters occupy 3 bytes in UTF-8 encoding. Slicing by bytes will result in garbled characters.

3.2 How to determine parameters?

Usually, we recommend: 500–800 words per chunk, with 50–100 words overlap. This is equivalent to about half to two-thirds of an A4 page, with adjacent chunks overlapping by one or two sentences, ensuring semantic coherence without fragmenting the information too much.

四、 The third problem: How to make AI "read and understand" these chunks?

Although the text is sliced, AI doesn't understand words; it only understands numbers. This requires a "translation" process—converting each text chunk into a long list of numbers, which is a vector (Embedding).

4.1 What is a vector? A layman's explanation

Imagine describing a person's location on a map requires two coordinates: "longitude and latitude." Describing the meaning of a sentence requires more coordinates—for example, 3,072 dimensions. Lining up these 3,072 numbers creates the "semantic coordinates" of the sentence.

• Sentences with close meanings will have closer 3,072-dimensional coordinates.
• Sentences with distant meanings will have larger coordinate differences.

4.2 Batch processing to save trips

Gemini provides a "batch embedding" feature—allowing one API call to convert dozens of text chunks into vectors simultaneously, saving time spent on repeated network requests.

func GetEmbeddings(texts []string) ([][]float32, error) {
    // Set 10-second timeout
    ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
    defer cancel()

    client, err := NewGeminiClient(ctx, apiKey)
    if err != nil {
        return nil, err
    }
    model := client.EmbeddingModel("gemini-embedding-001")

    batch := model.NewBatch()
    for _, text := range texts {
        batch.AddContent(genai.Text(text))
    }

    // Embed all texts in one call
    res, err := model.BatchEmbedContents(ctx, batch)
    if err != nil {
        return nil, err
    }
    
    var embeddings [][]float32
    for _, item := range res.Embeddings {
        embeddings = append(embeddings, item.Values)
    }
    return embeddings, nil
}

4.3 Don't forget to set an "alarm clock"

Calling third-party AI APIs is not 100% guaranteed to succeed. Set a 10-second timeout—if there is no response within 10 seconds, abort and retry, or tell the user "Network error, please try again." This is far better than letting the user wait indefinitely staring at a loading animation.

V. The fourth problem: Where to store vectors? How to find them?

Vectors need to be stored so they can be retrieved when the user asks a question.

5.1 PostgreSQL is enough

There are specialized "vector databases" on the market (such as Milvus, Pinecone, etc.), but for most enterprises, using PostgreSQL with the pgvector extension is sufficient. The reasons are threefold:

1. One less service to maintain: The project already needs a relational database, so there is no need to set up a separate vector database.
2. Ensure data consistency (Transactions): It guarantees that "writing 50 vector chunks" and "updating document status to processed" either both succeed or both roll back.
3. Simple backup: All business data and vector data can be backed up together with a single command.

// Transaction assurance: chunk insertion and status update either both succeed or roll back
db.Transaction(func(tx *gorm.DB) error {
    // Batch write chunks
    if err := tx.CreateInBatches(&chunks, 50).Error; err != nil {
        return err
    }
    // Update document status
    if err := tx.Model(&doc).Update("status", "done").Error; err != nil {
        return err
    }
    return nil
})

5.2 How to find the most similar chunks?

When a user asks a question, we first convert the question's text into a vector, and then search the database for the 3–5 chunks "closest" to it:

// SearchRelevantChunks finds the chunks most similar to the query vector
func SearchRelevantChunks(query string, topK int) ([]ChunkModel, error) {
    // Convert query to vector first
    queryVectors, err := GetEmbeddings([]string{query})
    if err != nil {
        return nil, err
    }
    queryVector := queryVectors[0]

    var chunks []ChunkModel
    // Order by Cosine Distance (<=>), smaller values mean more similar
    db.Order("embedding <=> ?", queryVector).Limit(topK).Find(&chunks)
    return chunks, nil
}

[!TIP]
Retrieving 3–5 chunks is a deliberate choice—too few might miss critical information, while too many will introduce irrelevant noise and may even exceed the AI's context window.

VI. The fifth problem: How to teach AI to "cite"?

After finding the relevant chunks, you cannot just dump them to the AI. If you let the AI express freely, it might start "hallucinating with a straight face"—this is what people commonly refer to as AI Hallucination.

6.1 Set "rules" for the AI

We need to assemble the retrieved knowledge base chunks into a prompt with sources and rules:

// Concatenate context: attach title and URL to each chunk
var contextText string
for i, chunk := range chunks {
    contextText += fmt.Sprintf(
        "[Source %d - Title: %s | Link: %s]\nContent: %s\n\n",
        i+1, chunk.Title, chunk.URL, chunk.Text,
    )
}

// Build complete Prompt
prompt := fmt.Sprintf(`
【Reference Materials】:
%s
【User Query】: %s
`, contextText, question)

After assembly, the prompt received by the AI looks like this:

【Reference Materials】:
[Source 1 - Title: Employee Handbook | Link: https://internal.corp/docs/1]
Content: Employees are entitled to 15 days of paid annual leave each year...

[Source 2 - Title: Attendance Policy | Link: https://internal.corp/docs/2]
Content: Leave requests must be submitted three days in advance...

【User Query】: What is the company's annual leave policy?

At the same time, we attach three strict rules in the system prompt:

1. Answer only based on the reference materials, absolutely do not make things up.
2. Crucial conclusions must cite sources (e.g., "According to the [Employee Handbook]...").
3. If you cannot find the answer, state that you don't know; do not explain forcefully.

By pairing this with a lower model temperature setting (reducing the AI's creativity), the AI will strictly stick to the materials, drastically reducing the hallucination rate.

VII. The sixth problem: How to prevent users from waiting too long?

AI generation typically takes 5–20 seconds for a complete answer. Letting the user stare at a blank loading icon for so long provides a terrible experience.

7.1 "Talk while thinking" streaming output

Gemini supports streaming output—like a person speaking while thinking, outputting words as they are generated. Whenever the backend receives a small piece of text, it immediately pushes it to the frontend via a channel or a callback function:

// AskStream core RAG streaming query function
func AskStream(ctx context.Context, question string, onChunk func(string)) error {
    // 1. Convert query to vector and search for the top 5 most relevant chunks
    queryVec := GetEmbedding(question)
    chunks := SearchTopK(queryVec, 5)
    
    // 2. Build Prompt
    prompt := BuildPrompt(chunks, question)
    
    // 3. Call Gemini streaming API, read chunk by chunk
    iter := model.GenerateContentStream(ctx, prompt)
    for {
        resp, err := iter.Next()
        if err == iterator.Done {
            break // Generation completed
        }
        if err != nil {
            return err
        }
        
        text := extractText(resp)
        if text != "" {
            onChunk(text) // Push a small chunk of text to the frontend
        }
    }
    onChunk("[DONE]") // Tell the frontend: finished
    return nil
}

This way, the frontend can render a smooth typewriter effect:

• 0.5s: Thinking...
• 1.0s: According
• 2.0s: According to the Employee Handbook,
• 3.0s: According to the Employee Handbook, employees are entitled to 15 days of paid annual leave each year...

The user's first reaction is no longer "Why is it so slow?", but "It's already writing the answer," significantly improving perceived speed.

7.2 Run it in the background with a separate Goroutine

Processing AI requests is a time-consuming CPU and network task. In Go, we usually spin up a separate Goroutine to call the AI API, avoiding blocking the main service's event loop and ensuring the overall throughput of the system.

VIII. Summary: Building AI applications is different from tuning AI models

To successfully turn AI capabilities into a stable product, engineering implementation capabilities are often more critical than the model itself.

The stability and experience of the entire RAG pipeline are built upon concrete engineering details:

• Stability comes from fine-grained handling of network timeouts and exception retries.
• Efficiency comes from the reasonable utilization of batch processing and concurrency.
• Credibility comes from strict instruction constraints on the AI and traceability requirements.

The best technology is using the most stable engineering to carry the smartest AI.