Build a local FAQ search with Ollama and EmbeddingGemma

Find the right passage before generating an answer

A customer asks how to share project documents. Your approved FAQ describes the workspace, but the wording differs from the question. Semantic retrieval can help locate that passage without asking a language model to invent a response. Start with a handful of approved answers and inspect what the search actually returns.

This AI-assisted editorial guide uses primary documentation checked September 27, 2026. The Python example has syntax and synthetic-vector checks; we have not downloaded or benchmarked the model for this article. Its sample FAQ is fictional demonstration content, not Sociologix service policy.

1. Prepare Ollama and the embedding model

Install Ollama from its official download page and open the application. On Linux, start ollama serve if the server is not already running. The local model path does not require an API key. This tutorial calls the loopback endpoint directly and uses no cloud model. [1]

Run the pull command below to download EmbeddingGemma. Its Ollama listing requires version 0.11.10 or later and currently lists a 2K context window. Keep FAQ entries short; a lengthy service manual needs deliberate splitting into useful passages. Review the linked model terms before adopting it in a commercial deployment. [2][3]

Ollama binds to 127.0.0.1:11434 by default. Keep the prototype local. A public website would need a separately designed server-side retrieval service with access controls; exposing your laptop’s model endpoint is not the next deployment step. [4]

ollama pull embeddinggemma

2. Preserve the source, not just the vector

Create a small approved corpus where every entry has a stable ID, a useful title and the actual answer text. For a real project, also retain a document location, review date and access classification. The numeric vector helps find a passage; the original source lets a reviewer decide whether it answers the question.

Google documents different input prefixes for retrieval queries and documents. The example explicitly formats titles and passages, then prefixes the question for search. If you later use a wrapper that inserts these prompts automatically, avoid adding them twice. [3]

Use the same embedding model for stored documents and questions. Ollama’s embedding guide supports batched inputs and recommends cosine similarity for semantic search. Rebuild your document vectors when you change the model or text rather than mixing incompatible versions. [5]

3. Rank three example passages with Python

Save this as faq_search.py. It uses Python’s standard library to send one batch containing the three documents and the question. The endpoint returns an embeddings array. Setting truncate to false makes an oversized input fail instead of silently shortening it. [6]

The script prints the top two candidates with their IDs and scores. It does not generate a new answer or make a customer-care decision. Keeping that boundary visible makes it easier to assess whether retrieval is useful before adding more behavior.

import json
import math
import sys
from urllib.request import Request, urlopen

MODEL = "embeddinggemma"
ENDPOINT = "http://127.0.0.1:11434/api/embed"
# Fictional demonstration content, not company policy.
FAQ = [
    {"id": "demo-brief", "title": "Preparing a project brief",
     "text": "Describe the workflow, its users and the intended result. "
             "Include a sample of the current process."},
    {"id": "demo-files", "title": "Sharing project files",
     "text": "Use the approved project workspace to share documents. "
             "Remove unrelated personal information before uploading."},
    {"id": "demo-scope", "title": "Changing project scope",
     "text": "Submit the proposed change for review before work begins. "
             "Agree on its delivery and pricing impact."},
]


def embed(texts):
    payload = {"model": MODEL, "input": texts, "truncate": False}
    request = Request(ENDPOINT, data=json.dumps(payload).encode("utf-8"),
                      headers={"Content-Type": "application/json"})
    with urlopen(request, timeout=60) as response:
        vectors = json.load(response)["embeddings"]
    if len(vectors) != len(texts):
        raise ValueError("Unexpected embedding count")
    return vectors


def cosine(a, b):
    if not a or len(a) != len(b):
        raise ValueError("Vectors must be nonempty and equally sized")
    if not all(math.isfinite(x) for x in a + b):
        raise ValueError("Non-finite vector value")
    denominator = math.sqrt(sum(x*x for x in a) * sum(x*x for x in b))
    if denominator == 0:
        raise ValueError("Zero-length vector")
    return sum(x*y for x, y in zip(a, b)) / denominator


def main():
    question = " ".join(sys.argv[1:]).strip()
    if not question:
        raise SystemExit('Usage: python faq_search.py "your question"')
    documents = [f"title: {row['title']} | text: {row['text']}" for row in FAQ]
    vectors = embed(documents + [f"task: search result | query: {question}"])
    query_vector = vectors[-1]
    ranked = sorted(
        [(cosine(query_vector, vector), row)
         for vector, row in zip(vectors[:-1], FAQ)],
        key=lambda item: item[0], reverse=True,
    )
    print("Candidate passages for review; scores are not confidence probabilities.")
    for score, row in ranked[:2]:
        print(f"{row['id']} | score={score:.4f} | {row['title']}")
        print(row["text"])


if __name__ == "__main__":
    main()

4. Evaluate matches, including questions with no answer

Run the example command after Ollama has loaded the downloaded model. Inspect the returned passages rather than assuming the first candidate is correct. The program always produces a ranking for a nonempty corpus, even when none of its passages answers the question.

A cosine score measures vector similarity, not the probability that an answer is correct. Do not copy an arbitrary acceptance threshold from another project. Build a labeled evaluation set from your actual service questions and review misses before choosing any automated acceptance rule.

python faq_search.py "Where should I send project documents?"

• Paraphrase: ask the same question in several natural ways and check whether the useful source remains among the top results.
• Unknown policy: ask for a refund deadline absent from the corpus. Record that no passage answers it, even though candidates appear.
• Ambiguity: ask about changing a project and inspect whether scope review is confused with file sharing.
• Policy revision: edit an answer, rebuild its embedding, and confirm the displayed passage contains the updated text.
• Permissions: test only sources the current user is entitled to read; local execution does not enforce your document permissions.

5. Move from an experiment to a maintained retrieval service

For this tiny example, recomputing everything on each run keeps the mechanics visible. In an application, store approved document vectors with their source metadata and embed only each incoming question. Update the index when content changes. Apply access restrictions before candidates can be exposed to a user.

If the connection is refused, check that Ollama is running. If the model is missing, complete the pull. If input exceeds the context limit, shorten or split the source and retry; do not quietly discard the paragraph that contains the policy exception. Stop on endpoint or malformed-vector errors rather than displaying an apparently valid result.

Before adding generated answers, define what happens when no approved passage is sufficient: ask a clarifying question or offer a human handoff. Measure retrieval relevance, response time on your hardware and the effort required to maintain the corpus. Those observations are more useful than claiming that a local model alone makes customer care autonomous.

Sources & further reading

1. Ollama: Quickstart and local setup
2. Ollama: EmbeddingGemma model listing
3. Google: EmbeddingGemma model card and retrieval prompts
4. Ollama: FAQ and local network binding
5. Ollama: Embeddings and semantic search
6. Ollama: Generate embeddings API

Make your service knowledge easier to use

Sociologix can help organize approved knowledge, evaluate retrieval and build customer-care assistants with clear escalation paths. Bring a sample FAQ and the questions your team struggles to answer consistently.