Transforming LLMs from "Chatting Tools" into Real Working "System Employees": Go + Function Calling in Practice

Preface

Recently, I've been upgrading the backend of my platform TapeCode to enable AI to truly participate in platform operations, rather than just acting as a fancy chat box on a webpage.

Ordinary AI conversations can answer questions, but they cannot do the following:

"Go to the database and check what courses this user has purchased."
"Ban this user who is farming orders or using abusive language in the comment area for 7 days."

AI Agent + Function Calling is the golden key to solve this problem.

In this article, combining real code from my project, I will clearly explain from scratch:

What is Function Calling, and why does it make AI capable of "doing work"?
How to abstract a general Go Agent execution engine?
How to implement two practical Agents in a real online education platform.

I. Ordinary LLM Call vs. Agent Call

First, clarify a cognitive gap:

Ordinary LLM Call: You send a sentence, and it replies with a sentence. It belongs to simple Q&A.
- You: "Which courses did this user buy?"
- AI: "I don't know. I don't have the permission or capability to access your database."
Agent + Function Calling Call: You tell the LLM “you can use these specific tools.” The LLM will autonomously decide whether to call them, which one to call, and what parameters to pass based on the context. Finally, the system executes the tool and returns the result to the LLM, which then gives the final answer.
sequenceDiagram actor User as Operator/Admin participant System as Go Backend (TapeCode) participant LLM as Doubao LLM (Doubao) User->>System: Check what courses user with ID 42 bought? System->>LLM: Request with available tools [query_user_orders] Note over LLM: Decision: Call query_user_orders<br/>Parameters: user_id=42 LLM-->>System: Trigger ToolCall System->>System: Query DB: ToolQueryUserOrders(42) System->>LLM: Return tool execution results [Order Data] LLM-->>System: Synthesize results and translate to natural language System-->>User: User 42 bought "Go Programming Introduction" and "Docker in Action", a total of two courses.

This is what it truly means to integrate AI into a business system.

II. LLM Client Wrapper

The project uses the Doubao LLM under ByteDance, accessed via the Volcano Engine ARK Runtime Go SDK.

2.1 Basic Conversation (Streaming In, Returning Full Result)

go Copy

// SendContent sends content to LLM (internally using stream, returning full result)
func SendContent(systemPrompt string, history []Message) (string, error) {
    client := NewLLMClient()
    
    // Construct message list, including System Prompt and history conversation
    reqMessages := []*model.ChatCompletionMessage{
        {
            Role:    model.ChatMessageRoleSystem,
            Content: &model.ChatCompletionMessageContent{StringValue: volcengine.String(systemPrompt)},
        },
    }
    for _, msg := range history {
        reqMessages = append(reqMessages, &model.ChatCompletionMessage{
            Role:    msg.Role,
            Content: &model.ChatCompletionMessageContent{StringValue: volcengine.String(msg.Content)},
        })
    }
    req := model.CreateChatCompletionRequest{
        Model:    modelID,
        Messages: reqMessages,
        Stream:   volcengine.Bool(true),
    }
    
    stream, err := client.CreateChatCompletionStream(context.Background(), req)
    if err != nil {
        return "", err
    }
    defer stream.Close()
    var fullContent strings.Builder
    for {
        resp, err := stream.Recv()
        if err == io.EOF {
            break
        }
        if err != nil {
            return "", err
        }
        delta := resp.Choices[0].Delta.Content
        if delta != "" {
            fullContent.WriteString(delta)
        }
    }
    return fullContent.String(), nil
}

2.2 Streaming Push (Used for real-time WebSocket output)

Using the callback function StreamCallback to push tokens to the frontend in real-time, achieving a typewriter experience similar to ChatGPT.

go Copy

type StreamCallback func(string)
// SendStreamContent triggers a callback every time a delta is received
func SendStreamContent(systemPrompt string, history []Message, onData StreamCallback) (string, error) {
    client := NewLLMClient()
    // ... Construct reqMessages logic same as above ...
    
    req := model.CreateChatCompletionRequest{
        Model:    modelID,
        Messages: reqMessages,
        Stream:   volcengine.Bool(true),
    }
    stream, err := client.CreateChatCompletionStream(context.Background(), req)
    if err != nil {
        return "", err
    }
    defer stream.Close()
    var fullContent strings.Builder
    for {
        resp, err := stream.Recv()
        if err == io.EOF {
            break
        }
        if err != nil {
            return "", err
        }
        delta := resp.Choices[0].Delta.Content
        if delta != "" {
            fullContent.WriteString(delta)
            onData(delta) // Push to frontend channel/WebSocket in real-time
        }
    }
    return fullContent.String(), nil
}

2.3 Conversation with Tools (Agent Core)

go Copy

// ChatWithTools calls LLM with tool definitions
func ChatWithTools(messages []*model.ChatCompletionMessage, tools []*model.Tool) (*model.ChatCompletionResponse, error) {
    client := NewLLMClient()
    req := model.CreateChatCompletionRequest{
        Model:    modelID,
        Messages: messages,
        Tools:    tools, // Key: Tell LLM the tools list and parameter descriptions
    }
    resp, err := client.CreateChatCompletion(context.Background(), req)
    if err != nil {
        return nil, err
    }
    return &resp, nil
}

III. General Agent Engine

In order to reuse Agent logic elegantly and avoid writing a batch of redundant checking code for every function, we abstracted an AgentConfig struct and a general RunAgent engine:

go Copy

// AgentConfig defines the configuration and capabilities of a "Skill Agent"
type AgentConfig struct {
    Name         string
    SystemPrompt string
    Tools        []*model.Tool
    ToolMap      map[string]func(string) string // Tool executor: function name -> local execution handler
}

The general execution engine RunAgent orchestrates the entire two-round call process:

go Copy

// RunAgent general Agent running engine
func RunAgent(agent AgentConfig, userQuery string) string {
    // Step 1: Construct conversation, inject System Prompt and user query
    messages := []*model.ChatCompletionMessage{
        {
            Role:    model.ChatMessageRoleSystem,
            Content: &model.ChatCompletionMessageContent{StringValue: volcengine.String(agent.SystemPrompt)},
        },
        {
            Role:    model.ChatMessageRoleUser,
            Content: &model.ChatCompletionMessageContent{StringValue: volcengine.String(userQuery)},
        },
    }
    // Step 2: First round call, LLM decides whether to use tools
    resp, err := ChatWithTools(messages, agent.Tools)
    if err != nil {
        return fmt.Sprintf("Error in LLM calling: %v", err)
    }
    msg := resp.Choices[0].Message
    // Step 3: If no tool call is needed, it means AI can reply directly, return content
    if len(msg.ToolCalls) == 0 {
        return *msg.Content.StringValue
    }
    // Step 4: LLM emits ToolCalls intent, Go maps/reflects it locally to perform real action
    messages = append(messages, &msg)
    for _, toolCall := range msg.ToolCalls {
        fname := toolCall.Function.Name
        args  := toolCall.Function.Arguments
        var result string
        if handler, ok := agent.ToolMap[fname]; ok {
            result = handler(args) // Call corresponding local business function
        } else {
            result = "Error: Tool not found"
        }
        // Append the actual execution result of the tool to the context as a "Tool" role message
        messages = append(messages, &model.ChatCompletionMessage{
            Role:       model.ChatMessageRoleTool,
            Content:    &model.ChatCompletionMessageContent{StringValue: volcengine.String(result)},
            ToolCallID: toolCall.ID,
        })
    }
    // Step 5: Second round call, LLM combines the first round context and the newly obtained tool results to output natural language
    finalResp, err := ChatWithTools(messages, agent.Tools)
    if err != nil {
        return fmt.Sprintf("Error in final LLM calling: %v", err)
    }
    return *finalResp.Choices[0].Message.Content.StringValue
}

IV. Two Real Business Agents

In the TapeCode platform, two business scenario Agents are currently implemented:

4.1 DataAgent: Data Query Expert

Responsibility: Provides rapid queries of user information and purchase records for the operations team.
Defined Functions/Tools:
- query_user_info: Query basic user information by ID or nickname.
- query_user_orders: Query the user's recent paid order records.

go Copy

func NewDataAgent() AgentConfig {
    return AgentConfig{
        Name:         "DataStats",
        SystemPrompt: cfg.DataAgentPrompt, // Read from config
        Tools: []*model.Tool{
            {
                Type: model.ToolTypeFunction,
                Function: &model.FunctionDefinition{
                    Name:        "query_user_info",
                    Description: "Query basic user information such as ID, nickname, email, etc. Input can be User ID or Nickname.",
                    Parameters: map[string]any{
                        "type": "object",
                        "properties": map[string]any{
                            "keyword": map[string]any{
                                "type":        "string",
                                "description": "User ID or Nickname",
                            },
                        },
                        "required": []string{"keyword"},
                    },
                },
            },
        },
        ToolMap: map[string]func(string) string{
            "query_user_info": func(argsStr string) string {
                var args struct{ Keyword string `json:"keyword"` }
                _ = json.Unmarshal([]byte(argsStr), &args)
                return ToolQueryUserInfo(args.Keyword)
            },
        },
    }
}
// Concrete underlying database query function
func ToolQueryUserInfo(keyword string) string {
    var user models.UserModel
    if err := global.DB.Preload("Roles").Where("id = ? OR nickname = ?", keyword, keyword).First(&user).Error; err != nil {
        return "User information not found"
    }
    return fmt.Sprintf("ID: %d\nNickname: %s\nEmail: %s\nRegistered At: %s",
        user.ID, user.Nickname, user.Email, user.CreatedAt.Format("2006-01-02"))
}

4.2 OpsAgent: Operations/Ops Expert

Responsibility: Automatically extract information and execute platform control operations, such as banning violating users.
Tool: banned_user (requires user ID, ban reason, and ban days).

go Copy

func NewOpsAgent() AgentConfig {
    return AgentConfig{
        Name:         "SysAdmin",
        SystemPrompt: cfg.OpsAgentPrompt,
        Tools: []*model.Tool{
            {
                Type: model.ToolTypeFunction,
                Function: &model.FunctionDefinition{
                    Name:        "banned_user",
                    Description: "Ban a violating user, requires User ID, ban reason, and ban days",
                    Parameters: map[string]any{
                        "type": "object",
                        "properties": map[string]any{
                            "user_id": map[string]any{"type": "string", "description": "User ID to ban"},
                            "reason":  map[string]any{"type": "string", "description": "Ban reason"},
                            "days":    map[string]any{"type": "integer", "description": "Ban days"},
                        },
                        "required": []string{"user_id", "reason", "days"},
                    },
                },
            },
        },
        ToolMap: map[string]func(string) string{
            "banned_user": func(argsStr string) string {
                var args struct {
                    UserID string `json:"user_id"`
                    Reason string `json:"reason"`
                    Days   int    `json:"days"`
                }
                _ = json.Unmarshal([]byte(argsStr), &args)
                userID, _ := strconv.ParseUint(args.UserID, 10, 32)
                return ToolBannedUser(uint(userID), args.Reason, args.Days)
            },
        },
    }
}
func ToolBannedUser(userID uint, reason string, days int) string {
    var user models.UserModel
    if err := global.DB.First(&user, userID).Error; err != nil {
        return "Ban operation failed: User not found"
    }
    if user.IsBanned {
        return fmt.Sprintf("User %s has already been banned", user.Nickname)
    }
    expiry := time.Now().AddDate(0, 0, days)
    global.DB.Model(&user).Updates(map[string]any{
        "is_banned":  true,
        "banned_at":  time.Now(),
        "ban_reason": reason,
        "ban_expiry": expiry,
    })
    return fmt.Sprintf("Successfully banned user %s, Reason: %s, Period: %d days", user.Nickname, reason, days)
}

[!CAUTION]
Safety Warning (Human-in-the-Loop): When dealing with high-risk operations such as "banning", "deleting", "exporting data", or "payment", never completely hand over the final decision-making power to AI. It is highly recommended to add a human confirmation button before ToolMap execution, letting humans make the final decision and having AI act as a "pre-processor" or "drafter".

V. Design Highlights Summary

System Prompt Configuration: We decouple the Agent's SystemPrompt from the code and store it in a configuration file. You can dynamically adjust the AI's work boundaries and "personality traits" without compiling and redeploying.
Strict Tool Parameters JSON Schema Definition: LLMs do not understand your code signatures; the only basis for them to judge parameter types and delivery specifications is the JSON Schema we fill in. Adding a clear description to each Tool is the foundation for high-precision calls.
Standard Paradigm of Two-Round Calls:
- First round: Pass the request to LLM, and the model decides whether to use tools, which one to use, and what parameters to pass.
- Second round: The system executes the call locally, returns the results to the model, and the model post-processes them to output natural language.

LLM Client Singleton Optimization:
Use sync.Once to ensure only one LLM RPC client is built globally, avoiding repeated HTTP connection requests under high concurrency:

go Copy

var (
    once   sync.Once
    client *arkruntime.Client
)
func NewLLMClient() *arkruntime.Client {
    once.Do(func() {
        client = arkruntime.NewClientWithApiKey(cfg.ArkApiKey, ...)
    })
    return client
}

VI. Core Security Mechanisms

Although LLMs are powerful, we must maintain absolute respect in production environments:

Principle of Least Privilege: The database account dedicated to the Agent should be strictly isolated. For example: DataAgent should only use a read-only database account under read-write separation to prevent dirty data caused by hallucinations.
Complete Operation Audit Log: Every tool call triggered by an Agent must write a complete Audit Log in the database (including original question, parameters extracted by LLM decision, and actual execution results) for subsequent accountability and logical back-tracing.
Rate Limiting: Prevent the LLM from falling into infinite loops in ReAct (Chain of Thought) mode or being subjected to malicious input attacks, causing malicious write shocks to the database service in a short period of time.

VII. Conclusion

The core idea of Function Calling is actually very simple: You teach the AI to know your local tools, and the AI itself decides when to use them.
And Go is naturally suitable for this kind of engineering-structural encapsulation: type safety, clear concurrency model, and easy integration into microservices and enterprise frameworks like GORM.
If you are also building smart backends with Go, hope this article inspires you!