Secure RAG on Coveo: A Practical Implementation Guide

What secure RAG on Coveo actually looks like

Secure RAG on Coveo starts with one claim you have to defend in a security review: the assistant never surfaces a document the person asking cannot already open. Everything else — chunking, prompts, model choice — sits downstream of that single guarantee.

Architecturally, three layers do the work. First, connectors pull content and its permissions out of each source system. Second, the index stores those permissions alongside the content. Finally, the retrieval layer filters on the caller’s identity before a single passage reaches a language model.

Coveo calls that middle layer early binding. The platform imports item permissions at crawling time, so it removes items a user cannot access before the query runs rather than filtering afterward. That ordering matters far more than any prompt engineering you will do later.

Step 1: Map the permission model of every source

Before you configure anything, inventory what you actually intend to index. In a typical enterprise, policies live in SharePoint, runbooks in Confluence, tickets in ServiceNow, and contracts on a file share. Each one enforces access differently.

For every source, write down three things: who grants access, at what granularity, and how fast changes propagate. Site-level permissions behave nothing like per-record ACLs. Consequently, a “unified index” that flattens both into one model will either over-share or under-serve.

This step looks administrative, so teams rush it. In practice, it holds most of the project risk — and skipping it is the single most common reason pilots stall for months.

Step 2: Index permissions as Coveo security identities

Coveo models access through security identities — granted identities, groups, and virtual groups — managed by security identity providers and held in a security identity cache. Each indexed item carries the permission sets its source system reported at crawl time.

Two failure modes show up here again and again. First, group expansion goes stale, so a departure or a role change takes days to reach the index. Second, nested or dynamic groups resolve incompletely, which quietly widens access without anyone noticing.

Therefore, treat identity refresh as a monitored production job, not a one-time setup task. Ask your partner for the real refresh interval. Then test it: change a live account’s group membership and confirm how long the index takes to agree.

Step 3: Ground the answer with Coveo Passage Retrieval

Retrieval is the layer where secure RAG on Coveo earns its name. Coveo Passage Retrieval (CPR) chunks indexed content into passages, then runs a two-stage retrieval. First, the search engine finds the most relevant items using keyword and vector search. Second, CPR pulls the most relevant passages out of those items.

Because stage one runs against an already permission-filtered result set, every passage that reaches your prompt inherits the caller’s access rights. In other words, the security boundary lives in retrieval — not in the model, and not in the prompt.

Two practical requirements catch teams out. CPR needs both a CPR model and a Semantic Encoder model attached to the query pipeline. It is also a paid product extension, so budget for it explicitly rather than discovering it mid-build.

RGA or the Passage Retrieval API: which one fits

You have two ways to generate the answer, and the choice is architectural rather than cosmetic.

Relevance Generative Answering (RGA) is the managed option. Coveo controls retrieval, builds the prompt, and calls a third-party LLM. According to Coveo’s RGA data security documentation, that model is stateless, is not trained on your enterprise data, and retains none of it. Coveo also states it does not retain your enterprise content after indexing.

The Passage Retrieval API is the build-your-own option. You POST a query to the passage retrieval endpoint and receive passages, each with a relevance score and document metadata. From there, you own the prompt, the model, and the hosting.

Choose RGA when time to value wins and a shared foundation model is acceptable. Choose the API when you need your own model, a specific hosting region, or an agent that does more than answer questions.

The bilingual gap in secure RAG on Coveo

Here is the detail that reshapes Canadian and European rollouts. Coveo’s documentation states that passage retrieval supports English content by default, with multilingual support available in beta.

For a bilingual organization, that one line changes the plan. Your French policies, French runbooks, and French collective agreements are not secondary content — they are frequently the documents employees search hardest for and find least often.

So test both languages inside the pilot, never after it. Measure French recall separately from English recall, and report them separately to your steering committee. Otherwise you ship an assistant that answers confidently in one language and shrugs in the other.

Nine questions your security review will ask

Bring answers to these before the review, not during it. Each one has sunk a project we were later asked to rescue.

1. What exactly does a contractor see when they query the assistant?
2. Which permission model does each connector import, and at what granularity?
3. How long after an offboarding does the index stop returning that person’s access?
4. Do nested and dynamic groups expand completely, and how do you prove it?
5. Does the generated answer cite only documents the caller can open?
6. Where is content processed, and which region hosts the model?
7. What does the LLM provider retain, and for how long?
8. How do you test permission correctness — manually, or as an automated suite?
9. What happens to the answer when a source system is temporarily unreachable?

Notice that only two of the nine touch the AI itself. The rest are access-control questions wearing an AI costume.

How Sengo delivers secure RAG on Coveo

Sengo builds secure RAG on Coveo from the permission model outward. We start with the source inventory, prove early-binding behaviour with real accounts, and only then wire up generation. As a result, the security review becomes a demonstration rather than a negotiation.

Our team includes a former Coveo backend developer, so security identities, indexing pipelines, and relevance tuning are familiar ground rather than documentation we read last week. We are also an official Coveo implementation partner, which means we advise and deliver.

We work in English and French natively, from Quebec. For bilingual enterprises, that removes the exact gap described above. Our enterprise delivery record includes iA Financial Group, Cirque du Soleil, FTQ, CCQ, and LCI Education.

If you want the vendor-neutral view first, read our breakdown of permission-aware enterprise search, or explore Sengo’s Coveo capabilities. When you are ready to scope the build, we will map your sources and give you an honest answer on effort.