Home Knowledge Base Knowledge Maintenance

Knowledge Maintenance

Last updated on Oct 21, 2024

Viewing Text Chunks

Each document you upload to the knowledge base is organized into segments known as text chunks. You can browse the detailed content of these chunks by accessing the chunks list.

Reviewing Chunk Quality

The effectiveness of your knowledge base's Q&A capabilities is heavily influenced by the quality of these document chunks. It's advisable to manually review the chunks for quality assurance before linking the knowledge base to any agent.

While automated chunking methods using character length, identifiers, or NLP semantic techniques can reduce labor for handling large text volumes, the intricacy of document formats and their semantic contexts necessitate manual checks to address machine chunk limitations in semantic processing.

Important factors to observe when assessing chunk quality include:

  1. Chunks that are too short, potentially losing semantic meaning.

  2. Chunks that are too lengthy, introducing semantic noise that may impact accuracy.

  3. Noticeable semantic truncation, which happens when exceeding segment length limits, causing forced truncation and omitting content during recall.

Adding Text Chunks

To include custom chunks in a document, navigate to the chunk list and click "Add a Chunk." This allows you to insert one or multiple chunks tailored to the document.

Editing Text Chunks

Within the chunk list, you can modify the added chunks by updating text content and keywords to ensure clarity and relevance.

Disabling Documents

You have the option to deactivate documents or specific chunks from being indexed in the dataset. To do so, locate the disable button in the dataset document list to deactivate the document. You can also choose to disable an entire document or a particular chunk within the document details. Once disabled, documents will not be indexed until you click enable to restore them.