A verifiable enterprise knowledge Agent
An end-to-end knowledge service that combines retrieval, Agent orchestration, document access control and inspectable citations. It is designed to make both the answer and the execution trace reviewable.
All companies, documents, questions and roles in the demo are fictional. Public visitors can ask questions, run Agent tasks and inspect cited evidence, sharing a daily quota of 10 requests. Document editing and memory writes are unavailable in the public demo. The API depends on the author's local server; recorded runs remain available independently.
System architecture
FastAPI exposes the knowledge and task APIs. PostgreSQL stores documents, versions, access rules, tasks and tool audit records. OpenSearch supplies BM25 and BGE-M3 vector retrieval, fused with reciprocal rank fusion. The current local generator is Qwen2.5 7B through Ollama; React and TypeScript provide the product interface.
LangGraph provides node scheduling, conditional transitions and execution checkpoints. Application code retains the business rules: tenant boundaries, live permissions, document version semantics, cumulative call budgets, evidence binding and audit. Resuming a checkpoint does not restore revoked access.
Choosing an execution strategy
| Strategy | Decision process | Current scope |
|---|---|---|
| RAG | Retrieve once, generate an answer and verify its citations. | Public knowledge Q&A. |
| Workflow | Code determines retrieval, bounded recovery, evidence preparation and stopping. | Default. Public visitor auto routing also follows this path. |
| Hybrid | The model proposes an evidence-seeking intent; code binds it to validated tool arguments. | Local experiment; not shown in the current recorded run. |
| Dynamic Agent | The model chooses subsequent tools and arguments within server-enforced limits. | Local experiment; no claim of superiority over Workflow. |
The public recording shows a real fixed Workflow with search_documents, retrieve_evidence, five evidence spans and a cited answer about API quotas. It does not demonstrate a version-comparison tool call or autonomous tool selection.
Reliability and access boundaries
- Authorization is applied inside retrieval and checked again when evidence is read or returned.
- Citations bind to real document versions and source locations. The UI opens source text or a PDF with parser-derived highlight coordinates.
- Historical and effective versions are handled explicitly; insufficient evidence and conflicting sources have distinct outcomes.
- Tool calls are audited and bounded. Cancellation takes effect at execution boundaries; stopping a browser wait does not terminate an ongoing model call.
Evaluation and engineering decisions
The constructed benchmark contains Dev 47, Core 70, Security 16 and External 150 tasks. Only Dev is used for tuning. The Core method and ablation matrix must be frozen before the first formal run; the headline metric is reviewed Core Strict Task Success. That result is currently pending.
The showcase separates historical development experiments and an older 80-question holdout from the current benchmark. Model-assisted reviews are labelled as such and are not independent human annotations. Test counts describe engineering validation, not task accuracy.
Earlier experiments motivated keeping reranking off by default, using rule-based follow-up context instead of an extra rewrite call, and reverting an ineffective topic-overlap check. These are historical decisions on specific task sets, not universal conclusions about those methods.
Current limitations
The system uses a local 7B model and a single-machine deployment without load-test evidence. Difficult multi-hop reasoning remains unreliable, and formal Core acceptance is incomplete. Scanned PDFs requiring OCR and PPTX are unsupported. Public visitors cannot use dynamic strategies or long-term memory.
For implementation details and historical experiments, see the technical report (Chinese) and the repository.
Overview prepared 10 October 2026 (Australia/Sydney). Capability descriptions reflect the implementation; availability and evidence limits are stated separately.