GIC Public Care — Multilingual Voice Assistant

🇱🇰 Multilingual Government Services RAG Assistant

A production-ready Retrieval-Augmented Generation (RAG) assistant that helps citizens access Sri Lankan government services in English, Sinhala, and Tamil, including support for romanized Sinhala (Singlish) input.

---

🌟 Key Features

🤖 Agentic AI Assistant

Unlike a traditional retrieve-and-answer pipeline, the assistant uses an LLM tool-calling loop to intelligently determine the best action for every query.

It can:

• 🔍 Search the knowledge base
• ⚙️ Execute transactional actions
• 👨‍💼 Escalate conversations to a human representative when needed

---

✅ Honest by Design

The system uses distance-gated retrieval to evaluate whether retrieved information is genuinely relevant before generating an answer.

If no reliable information is found, it will:

• Clearly state that it cannot answer confidently
• Avoid hallucinating incorrect information
• Offer to connect the user with a human representative

---

🌐 Multilingual Understanding

Supports:

• 🇬🇧 English
• 🇱🇰 Sinhala
• 🇮🇳 Tamil
• ✍️ Romanized Sinhala ("Singlish")

Queries are automatically normalized and language-mapped, allowing users to ask questions naturally regardless of the language used in the knowledge base.

---

🎙️ Real-Time Voice Conversations

Provides natural voice interactions through a streaming WebSocket pipeline.

Features include:

• Browser-side Voice Activity Detection (VAD)
• Speech-to-Text (STT)
• Streaming Text-to-Speech (TTS)
• Low-latency conversational experience

---

⚡ Interruptible Voice (Barge-In)

The microphone remains active even while the assistant is speaking.

If the user interrupts:

• Audio playback stops immediately
• Ongoing LLM generation is cancelled
• The assistant immediately begins processing the new request

This creates a much more natural conversational experience.

---

🔍 Retrieval Pipeline

A hybrid retrieval architecture designed for high accuracy.

Hybrid Search

• FAISS — Dense semantic vector search
• BM25 — Traditional keyword search

Re-ranking

Retrieved results are refined using a multilingual Cross-Encoder Re-ranker, significantly improving retrieval precision before passing context to the LLM.

Cross-Language Retrieval

User queries are:

• Normalized
• Language-mapped
• Matched against English knowledge base documents

This enables Sinhala, Tamil, and Singlish queries to successfully retrieve relevant English documentation.

---

🎤 Voice Processing Pipeline

Responses are streamed instead of waiting for the entire completion.

Streaming Response Generation

• LLM begins generating tokens immediately.
• Each completed sentence is streamed to the TTS engine.
• Audio playback begins after approximately the first sentence rather than waiting for the full response.

This dramatically reduces perceived latency and creates a smoother, more natural conversation.

---

🏗️ Technology Stack

Backend

• FastAPI
• LangChain
• FAISS
• Sentence Transformers
• Gemini API / Groq API

Frontend

• React
• Vite

---

☁️ Deployment

Backend

• Docker
• Google Cloud Run
• Google Cloud Build
• Google Secret Manager

Deployment Region: Mumbai (India)

Chosen to reduce latency by approximately 200 ms for users in Sri Lanka.

Frontend

• Vercel

---

🚀 Architecture Highlights

• 🌍 Production-ready multilingual RAG pipeline
• 🤖 Agentic LLM with tool calling
• 🔍 Hybrid semantic + keyword retrieval
• 📊 Multilingual Cross-Encoder re-ranking
• ✅ Distance-based hallucination prevention
• 🎙️ Streaming voice conversations
• ⚡ Interruptible (barge-in) interactions
• 🐳 Portable Docker deployment
• 🚀 One-command deployment to any Docker-compatible host