AIAI Smart Summary (TL;DR)
Collapse

A user says to the microphone, "Remind me to have a meeting tomorrow morning at eight."

Then Gemini says "Got it," and the conversation continues.

At exactly eight o'clock the next morning, a voice actually sounds out from the speaker saying, "It's time for the meeting."

What actually happens in between?

While building my own private AI assistant, I implemented this reminder mechanism. The entire link runs all the way from the microphone to the speaker, passing through speech recognition, tool calling, database, Redis events, WebSocket, and finally back to Gemini Live's speech synthesis—yet without a single line of polling code, and without introducing any task queue frameworks.

Let's trace this path from start to finish, explaining why each step was done this way and what pitfalls to watch out for.


Background: The Basic Architecture of This System

Let's clarify the macro background first, otherwise a lot of things won't have context later.

This AI assistant uses the Gemini Live API, an end-to-end full-duplex real-time voice interface. It is not a three-stage spliced pipeline of "STT → LLM → TTS", but a native speech model: users feed PCM audio streams directly in, and it returns PCM audio streams directly, with no independent speech recognition or speech synthesis steps in the middle.

The frontend uses the Web Audio API to capture microphone input at a 16kHz sampling rate, 16-bit PCM, and continuously sends it to the backend via WebSocket. The backend immediately forwards it to the Gemini Live long connection. When Gemini speaks, it returns a 24kHz PCM stream, which the backend then pushes to the frontend for playback via WebSocket.

The entire system looks roughly like this:

复制代码
Frontend Microphone (16kHz PCM)
    ↕  WebSocket
Go Backend
    ↕  HTTP/2 Long Connection
Gemini Live API

In this architecture, Gemini can not only speak, but also call tools—the backend pre-registers a batch of functions (checking time, storing memory, searching the web, setting reminders, etc.). After understanding the user's intent, Gemini actively initiates a function call request, the backend executes it and feeds the result back, and Gemini then tells the user the result via voice.

The reminder feature is plugged in through this tool-calling mechanism.


The First Hop: Voice Becomes a Tool Call

The user says "Remind me to have a meeting tomorrow morning at eight." This sentence is continuously sent to the backend in 16kHz PCM format via WebSocket, and the backend directly forwards it to the Gemini Live real-time connection.

Gemini Live listens and understands simultaneously, without needing to wait for the user to finish speaking before processing. When it recognizes the "reminder" intent from the voice, it makes an internal decision: this requirement corresponds to the setReminder tool I registered, and then it initiates a function call request, whose structure looks roughly like this:

json 复制代码
{
  "name": "setReminder",
  "args": {
    "content": "meeting",
    "remind_at": "2026-10-07 08:00:00"
  }
}

The backend's receive loop listens to the return stream of Gemini Live. Upon receiving the tool call request, it looks up the registered tool function and executes it directly, then feeds the result back into Gemini's conversation context. This entire process runs in an independent goroutine and does not block the reception of the main audio stream.

After Gemini receives the tool execution result ("Reminder set successfully"), it tells the user via voice, "Okay, I'll remind you tomorrow at eight." The user hears the response, and the whole thing feels that simple from the user's perspective.

But on the backend side, the real work inside the tool function has just begun.


The Second Hop: Writing to Two Places

When the setReminder tool function is invoked, it needs to record this reminder and arrange how to trigger it when the time comes.

The most straightforward approach is to write to a database, and then start a scheduled task to scan it every minute to see if there are any due reminders. Many projects do it this way. The drawback is that the precision is at best down to the minute, and the scanning frequency is inversely proportional to the table size—the larger the table, the slower it gets. Even with indexes, scanning hundreds of thousands of pieces of data every minute is still wasteful I/O.

I used another solution: Redis key expiration events.

The idea is simple: write a key with a TTL in Redis, where the TTL is the number of seconds until the reminder time. As soon as the key expires, Redis automatically pushes a message to a Pub/Sub channel, and the backend subscribes to this channel. When it receives the message, it triggers the reminder logic. There is no polling throughout the process; Redis actively pushes when it's time to act.

However, there is a detail to handle here: after the key expires, the data on that key disappears along with it. If the reminder content is stored directly on that expiring key, by the time the expiration event triggers, the data is already gone and nothing can be retrieved.

Therefore, we actually write to two places with clear divisions of labor:

go 复制代码
func AddReminder(ctx context.Context, userID int, content string, remindAt time.Time) (*Reminder, error) {
    now := time.Now()
    ttl := remindAt.Sub(now)
    if ttl <= 0 {
        return nil, fmt.Errorf("reminder time cannot be earlier than current time")
    }

    // First write: write to database for persistence; this is the ultimate source of truth
    rem, err := db.Reminder.Create().
        SetUserID(userID).
        SetContent(content).
        SetRemindAt(remindAt).
        SetIsSent(false).
        Save(ctx)
    if err != nil {
        return nil, fmt.Errorf("failed to write to database: %w", err)
    }

    // Second write: snapshot core fields into Redis Hash (never expires, pure in-memory)
    // key: "reminder:data", field: "reminder ID", value: JSON
    payload, _ := json.Marshal(ReminderPayload{
        ID:      rem.ID,
        UserID:  userID,
        Content: content,
        RemindAt: remindAt.Format(time.DateTime),
    })
    redisClient.HSet(ctx, "reminder:data", strconv.Itoa(rem.ID), payload)

    // Third write: write a pure timing empty key, triggering an expiration event when TTL expires
    // key: "reminder:timer:42", value is an empty string, we only care when it disappears
    timerKey := fmt.Sprintf("reminder:timer:%d", rem.ID)
    redisClient.Set(ctx, timerKey, "", ttl)

    return rem, nil
}

The reminder:data Hash does not expire and stores the actual reminder content; the reminder:timer:{id} String key is a countdown bomb whose TTL exhausts and automatically disappears, triggering an expiration event the moment it vanishes.

Each of the two mechanisms plays its own role without interfering with each other.


The Third Hop: Redis Expiration Events

Redis has a feature called Keyspace Notifications, which is disabled by default and needs to be manually enabled. Once enabled, whenever any key expires, Redis pushes a message to the __keyevent@{db}__:expired Pub/Sub channel, with the content being the name of that key.

When the backend starts up, it subscribes to this channel and waits indefinitely:

go 复制代码
func StartReminderListener(ctx context.Context) {
    // Automatically enable keyspace expiration notifications, sparing the trouble of manually configuring redis.conf
    redisClient.ConfigSet(ctx, "notify-keyspace-events", "Ex")

    // Use pattern subscription to match expiration events across all databases
    pubsub := redisClient.PSubscribe(ctx, "__keyevent@*__:expired")
    defer pubsub.Close()

    ch := pubsub.Channel()
    log.Info("Redis expiration event listener is ready")

    for {
        select {
        case <-ctx.Done():
            return
        case msg, ok := <-ch:
            if !ok {
                return
            }
            // All key expirations in Redis are pushed here; filter out our own
            if !strings.HasPrefix(msg.Payload, "reminder:timer:") {
                continue
            }
            // Parse reminder ID from the key name
            idStr := strings.TrimPrefix(msg.Payload, "reminder:timer:")
            id, err := strconv.Atoi(idStr)
            if err != nil {
                continue
            }
            // Process asynchronously to avoid blocking the listening loop
            go handleExpired(id)
        }
    }
}

This listener runs in an independent goroutine, completely isolated from the business logic. It doesn't know what the reminder content is, nor does it care how to notify the user. It is responsible for only one thing: telling the backend "Reminder No. 42 is due."


The Fourth Hop: Fetching Data, Two-Tier Fallback

After knowing the reminder ID, you need to find the corresponding content so you can tell the user what to do.

The fastest route is Redis Hash—it lives right in memory, and a single HGET has almost zero latency. But Redis is not permanent storage; it might restart, or run out of memory and trigger LRU eviction. If you rely solely on Redis, once the data is gone, the reminder will silently fail, and the user will receive nothing.

Therefore, there are two tiers: check Redis first, and if it's missing, check the database.

go 复制代码
func handleExpired(reminderID int) {
    ctx := context.Background()
    idStr := strconv.Itoa(reminderID)

    var payload ReminderPayload

    // First priority: check Redis Hash, retrieve directly from memory with millisecond latency
    raw, err := redisClient.HGet(ctx, "reminder:data", idStr).Result()
    if err == nil && raw != "" {
        json.Unmarshal([]byte(raw), &payload)
    }

    // Fallback: Redis has no data, degrade to querying the database
    // This is defensive design; under normal circumstances, data should exist in Redis
    if payload.UserID == 0 {
        rem, err := db.Reminder.Query().
            Where(
                reminder.IDEQ(reminderID),
                reminder.IsSentEQ(false), // Idempotency protection: do not re-process sent items
            ).
            WithUser().
            Only(ctx)
        if err != nil {
            log.Warn("Reminder triggered but data not found", "id", reminderID, "err", err)
            return
        }
        payload = ReminderPayload{
            ID:      rem.ID,
            UserID:  rem.Edges.User.ID,
            Content: rem.Content,
        }
    }

    log.Info("Reminder expired and triggered", "id", payload.ID, "content", payload.Content)

    // Trigger external notification (how to notify specifically is not decided in this package)
    if Notifier != nil {
        Notifier(uint(payload.UserID), payload.ID, payload.Content)
    }

    // Cleanup: mark database status as sent, clean up Redis Hash
    db.Reminder.UpdateOneID(payload.ID).
        SetIsSent(true).
        SetSentAt(time.Now()).
        Exec(ctx)
    redisClient.HDel(ctx, "reminder:data", idStr)
}

The database query adds the condition IsSentEQ(false), which is an idempotency protection—under extreme circumstances (such as a backlog of expired events during a restart), the same reminder might be triggered twice. This condition ensures that already-sent items are not processed repeatedly.


The Fifth Hop: Function Pointer Injection, Breaking Circular Dependencies

Pay attention to the Notifier variable in the code above; it is a function pointer defined in the reminder package:

go 复制代码
// reminder package
type NotifierFunc func(userID uint, reminderID int, content string)

var Notifier NotifierFunc

func SetNotifier(fn NotifierFunc) {
    Notifier = fn
}

Why go through all this trouble instead of calling the WebSocket push function directly inside handleExpired?

Because package dependency rules do not allow it.

The ws package is responsible for pushing WebSocket messages. Since the ws package manages Gemini Live sessions, it references the genkit package. When the genkit package registers tool functions, it needs reminder's AddReminder.

Visualized, it looks like this:

复制代码
ws → genkit → reminder

If we let reminder → ws as well, it forms a cycle: reminder → ws → genkit → reminder. The Go compiler has zero tolerance for circular dependencies and will throw an error immediately, no negotiation.

The solution is to abstract the decision-making power of "how to notify" away from the reminder package. The reminder package only defines a function pointer type, declaring "I need someone to handle notifications," but doesn't care who it is or how it's handled.

Then, when the ws package starts up, the specific pushing logic is injected in:

go 复制代码
// ws package, executed when Hub starts
func (m *Manager) bindReminderNotifier() {
    reminder.SetNotifier(func(userID uint, reminderID int, content string) {
        // Path 1: Push WebSocket message to frontend interface
        wsMsg := WSMessage{
            Type: "reminder_notify",
            Data: map[string]any{
                "id":      reminderID,
                "content": content,
                "ts":      time.Now().UnixMilli(),
            },
        }
        if sent := m.SendToUser(userID, wsMsg); !sent {
            log.Warn("User is offline, WebSocket notification not delivered", "uid", userID)
        }

        // Path 2: If the user is in a voice conversation, let Gemini speak it out loud
        go m.notifyLiveSession(userID, content)
    })
}

The dependency direction remains unidirectional: ws knows about reminder, but reminder knows about no one. A function pointer here is much less hassle than defining an interface and then implementing it—no extra type definitions, just one line of code for injection and you're done.


The Sixth Hop: WebSocket Push to Frontend

SendToUser finds all current online WebSocket connections for this user and stuffs them into the sending channel one by one. Each Client has a buffered Send chan []byte, and the write goroutine actually writes the data out from the channel.

go 复制代码
func (m *Manager) SendToUser(userID uint, msg any) bool {
    // Take a snapshot first to avoid holding locks for too long
    m.Lock.RLock()
    devices := m.Clients[userID]
    snapshot := make([]*Client, 0, len(devices))
    for _, c := range devices {
        snapshot = append(snapshot, c)
    }
    m.Lock.RUnlock()

    if len(snapshot) == 0 {
        return false
    }

    data, _ := json.Marshal(msg)
    sent := false
    for _, client := range snapshot {
        select {
        case client.Send <- data:
            sent = true
        default:
            // Send buffer is full, this message has to be dropped
            // For a personal system, this rarely happens; just log it
            log.Warn("Send buffer full, dropping message", "uid", userID, "device", client.DeviceID)
        }
    }
    return sent
}

The frontend receives the reminder_notify message type and pops up a reminder notification.

However, there is a scenario that frontend notifications cannot cover: the user is wearing headphones, engaged in an AI voice conversation, the interface is minimized, and the screen might even be locked. At this point, a silent WebSocket message cannot attract any attention at all.


The Final Hop: Letting Gemini Speak It Out Loud

If the user currently has an active Gemini Live real-time voice session, the most natural notification method is not to pop up a message box, but to let Gemini speak directly.

go 复制代码
func (m *Manager) notifyLiveSession(userID uint, content string) {
    m.Lock.RLock()
    devices := m.Clients[userID]
    var sessionIDs []int64
    for _, client := range devices {
        if client.SessionID > 0 {
            sessionIDs = append(sessionIDs, client.SessionID)
        }
    }
    m.Lock.RUnlock()

    for _, sessID := range sessionIDs {
        ls, ok := liveSessionStore.Get(sessID)
        if !ok {
            continue
        }

        // Inject a "system message" into Gemini Live's conversation context
        // After receiving it, Gemini will decide how to speak and voice it out in its own style
        err := ls.SendClientContent(LiveClientContentInput{
            Turns: []Content{{
                Role:  "user",
                Parts: []Part{{Text: fmt.Sprintf("Reminder: %s", content)}},
            }},
            TurnComplete: true, // Tell Gemini this message is finished and it can respond now
        })
        if err != nil {
            log.Warn("Failed to trigger Gemini Live voice broadcast", "session", sessID, "err", err)
        }
        break // Finding one active session is enough
    }
}

TurnComplete: true is the key parameter here. It means: this message ends right here, no more content will be appended, and Gemini, you can respond now. Without this, Gemini would think the user is still speaking, continue waiting, and say nothing.

After Gemini receives this injected message, it organizes its language in its own style and speaks it out via voice. There is no independent TTS module in the entire system—the AI's voice output is the 24kHz PCM stream directly returned by Gemini Live, sharing the exact same voice and texture as regular conversations.


The Complete Link, Viewed Together

Putting all six hops together:

复制代码
[User Speaks]
Microphone 16kHz PCM
    ↓ WebSocket binary frame
Go backend forwards to Gemini Live long connection
    ↓ Gemini recognizes intent, initiates tool call
setReminder tool function executes
    ├─→ Writes to PostgreSQL (persistence, source of truth)
    ├─→ Writes to Redis Hash (metadata snapshot, in-memory speed)
    └─→ Writes to Redis SETEX (countdown bomb, triggers upon TTL expiration)

[N minutes later, TTL exhausts]

Redis key expires
    ↓ __keyevent expired Pub/Sub pushed out
Go listening goroutine receives event
    ↓
handleExpired(id)
    ├─ Query Redis Hash → Get reminder content (normal path)
    └─ Fallback query PostgreSQL → Backup (when Redis data is lost)
    ↓
Call reminder.Notifier (function pointer callback, decouples circular dependency)
    ├─→ WebSocket push → Frontend interface pops notification
    └─→ Check if there is an active Gemini Live session
            ↓ Yes
        SendClientContent → Gemini determines how to respond
            ↓
        Gemini speaks, 24kHz PCM stream → WebSocket → Frontend playback

Three Design Decisions

Why not use a cron job to periodically scan tables?

Scanning once a minute gives a precision of at most ±60 seconds. As the table grows, the scanning cost goes up. The trigger precision of Redis expiration events is typically within 1 second, which is completely sufficient for a personal assistant, and it incurs zero extra I/O overhead.

Why split Hash and SETEX into two keys?

Because when a key expires, the data on that key vanishes along with it. If the reminder content were stored directly on that timing key, the data would already be gone upon triggering, and nothing could be retrieved. Splitting "countdown" and "data" into two independent keys keeps responsibilities clear and non-interfering.

Why not introduce a task queue framework (like Asynq, Machinery)?

This is a personal system, single-machine single-instance, with no high-concurrency scenarios. Introducing a task queue means more deployment dependencies, more complex failure retry logic, and longer troubleshooting chains. The Redis keyspace notification solution has less than 150 lines of core code, and it covers scenarios well enough. We can talk about scaling when we outgrow this.


The most interesting part of this link to me is the final hop: injecting a text message into the conversation context of Gemini Live, and letting Gemini speak it out loud itself. No TTS, no voice templates, and the voice texture is identical to regular conversations because it's literally the same model doing the talking.

Comments & Discussions

Write a comment...
0 / 500
Comments & Discussions (0)
暂无评论,快来抢占第一条沙发吧~
3
0
AIAI