Enterprise RAG in Practice: Six Steps Between a Pile of Documents and Usable Answers
栏目:技术洞察发布于:2026-10-02浏览:10792评论:0#Technology#RAG
Many enterprises' knowledge-base Q&A "can run, but isn't usable". The problem is basically in the six steps below; check them one by one and you can locate it — model selection only sets the ceiling, these six steps set the floor. Most projects' problem is a low floor.
1. Document parsing: lose the structure and everything after is wasted
Symptom: table content answered wrong, clause numbers don't match, text in scans can't be found.
Root cause: tables in PDF, text in scans, and directory structure in Word are discarded at parse time, leaving only a string of broken lines.
Practice: at parsing, retain heading levels, clause numbers, table row/column relationships, and section hierarchy; scans go through OCR with recognition-quality sampling; Excel-type tables are processed row-by-row, not converted to text.
Verify: randomly sample 10 original pages, compare whether parsed paragraphs and tables are complete; don't judge by "looks fine".
2. Chunking: by structure, not by character count
Symptom: the answer says half, different parts of the same clause contradict each other.
Root cause: hard chunking by fixed characters cuts a safety regulation
Practice: use clauses, sections, and Q&A pairs as the minimum unit; keep moderate contextual overlap for long sections; tag chunks with metadata (file name, section, effective date, scope, classification) so these tags participate in retrieval filtering.
Verify: check whether the same clause is split across chunks, and whether cross-chunk answers are complete.
3. Retrieval: vector + keyword, rerank when needed
Symptom: "what does clause 3.2 say" can't be found; "what to do before startup" answers off-topic.
Root cause: pure vector retrieval stumbles on technical terms, codes, abbreviations; pure keyword retrieval can't handle colloquial expressions.
Practice: hybrid recall (vector + keyword) covers both query types; add a rerank layer when needed to lift top-k quality; maintain a synonym table (abbreviations, colloquial names, English acronyms) for industry terms.
Verify: prepare 20 precise-type and 20 colloquial-type test questions respectively, and count recall hit rate separately.
4. Generation and citation: allow declining, force citations
Symptom: the answer looks reasonable but has no evidence, or it fabricates one when the library is empty.
Practice: three hard constraints — answer only from recalled content, every answer carries a citation (file name + section + original excerpt) for jump-verify, and decline with "not in library" when it can't answer.
Verify: construct 20 questions the library truly lacks, and check whether all are correctly declined. A zero decline rate is not good — it often means fabrication.
5. Permissions: filter before recall, not mask after
Symptom: those who should see can't, those who shouldn't see everything.
Practice: control visible scope by department, role, classification; out-of-scope content does not enter recall; retrieval happens within permission scope.
Verify: ask the same sensitive question with different-role accounts, compare recall and answer differences.
6. Updates: write the mechanism into operations, not patch after problems
Symptom: accurate at launch, starts erroring after half a year — the policy file changed, the knowledge base didn't.
Practice: clarify who feeds new documents, who takes down obsolete ones, how often a full check runs; support incremental updates and version management.
Verify: check whether policies revised in the last three months are synced in the knowledge base.
Three things to do before launch
- Build a baseline question set: the business unit provides 50–100 real questions with labeled standard answers as the acceptance baseline. Without a baseline, "is it good" can only be felt;
- Gray-scale trial: give a small team two weeks first, collect real questions and down-vote feedback, then open up;
- Feedback closure: the answer page must have a "answered wrong" entry, and someone handles it. When a user finds a wrong answer unanswered once, there won't be a second use.
One-page checklist
- Parsing: are table and clause structures retained? Are scans OCR'd and sampled?
- Chunking: chunked by structure? Is metadata complete?
- Retrieval: is hybrid recall on? Are term synonyms maintained?
- Generation: forced citation? Can it correctly decline?
- Permissions: filtered before recall?
- Updates: is the responsible person and frequency clear?
- Metrics: baseline set, gray-scale plan, feedback entry in place?
评论(0)
还没有评论,来说两句吧
