Enterprise RAG in Practice: Six Steps Between a Pile of Documents and Usable Answers

栏目:技术洞察发布于:2026-10-02浏览:10792评论:0#Technology#RAG

Many enterprises' knowledge-base Q&A "can run, but isn't usable". The problem is basically in the six steps below; check them one by one and you can locate it — model selection only sets the ceiling, these six steps set the floor. Most projects' problem is a low floor.

Six steps of RAG: parsing, chunking, retrieval, generation, permissions, updates

1. Document parsing: lose the structure and everything after is wasted

Symptom: table content answered wrong, clause numbers don't match, text in scans can't be found.

Root cause: tables in PDF, text in scans, and directory structure in Word are discarded at parse time, leaving only a string of broken lines.

Practice: at parsing, retain heading levels, clause numbers, table row/column relationships, and section hierarchy; scans go through OCR with recognition-quality sampling; Excel-type tables are processed row-by-row, not converted to text.

Verify: randomly sample 10 original pages, compare whether parsed paragraphs and tables are complete; don't judge by "looks fine".

2. Chunking: by structure, not by character count

Symptom: the answer says half, different parts of the same clause contradict each other.

Root cause: hard chunking by fixed characters cuts a safety regulation

RAG six-step process and step-by-step verification method

in half, and neither half answers accurately.

Practice: use clauses, sections, and Q&A pairs as the minimum unit; keep moderate contextual overlap for long sections; tag chunks with metadata (file name, section, effective date, scope, classification) so these tags participate in retrieval filtering.

Verify: check whether the same clause is split across chunks, and whether cross-chunk answers are complete.

3. Retrieval: vector + keyword, rerank when needed

Symptom: "what does clause 3.2 say" can't be found; "what to do before startup" answers off-topic.

Root cause: pure vector retrieval stumbles on technical terms, codes, abbreviations; pure keyword retrieval can't handle colloquial expressions.

Practice: hybrid recall (vector + keyword) covers both query types; add a rerank layer when needed to lift top-k quality; maintain a synonym table (abbreviations, colloquial names, English acronyms) for industry terms.

Verify: prepare 20 precise-type and 20 colloquial-type test questions respectively, and count recall hit rate separately.

4. Generation and citation: allow declining, force citations

Symptom: the answer looks reasonable but has no evidence, or it fabricates one when the library is empty.

Practice: three hard constraints — answer only from recalled content, every answer carries a citation (file name + section + original excerpt) for jump-verify, and decline with "not in library" when it can't answer.

Verify: construct 20 questions the library truly lacks, and check whether all are correctly declined. A zero decline rate is not good — it often means fabrication.

5. Permissions: filter before recall, not mask after

Symptom: those who should see can't, those who shouldn't see everything.

Practice: control visible scope by department, role, classification; out-of-scope content does not enter recall; retrieval happens within permission scope.

Verify: ask the same sensitive question with different-role accounts, compare recall and answer differences.

6. Updates: write the mechanism into operations, not patch after problems

Symptom: accurate at launch, starts erroring after half a year — the policy file changed, the knowledge base didn't.

Practice: clarify who feeds new documents, who takes down obsolete ones, how often a full check runs; support incremental updates and version management.

Verify: check whether policies revised in the last three months are synced in the knowledge base.

Three things to do before launch

  1. Build a baseline question set: the business unit provides 50–100 real questions with labeled standard answers as the acceptance baseline. Without a baseline, "is it good" can only be felt;
  2. Gray-scale trial: give a small team two weeks first, collect real questions and down-vote feedback, then open up;
  3. Feedback closure: the answer page must have a "answered wrong" entry, and someone handles it. When a user finds a wrong answer unanswered once, there won't be a second use.

One-page checklist

  • Parsing: are table and clause structures retained? Are scans OCR'd and sampled?
  • Chunking: chunked by structure? Is metadata complete?
  • Retrieval: is hybrid recall on? Are term synonyms maintained?
  • Generation: forced citation? Can it correctly decline?
  • Permissions: filtered before recall?
  • Updates: is the responsible person and frequency clear?
  • Metrics: baseline set, gray-scale plan, feedback entry in place?
分享到:微博

评论(0)

还没有评论,来说两句吧