Datasets

The datasets contain the information from the sources that you want to use with CTERA Content Services.

To add a dataset:

1. Select Datasets in the navigation pane.
   
2. Click New Dataset.
   
3. Enter the dataset details:
   • Dataset Details
     Name – A name to identify the dataset.
     Source – A source from the drop-down list. The source list includes the sources defined in Sources.
   • Capabilities
     The search and tagging capabilities to apply to this dataset.
     Enable Text Search – Enable keyword-based indexing to match exact terms, phrases, and specific identifiers. This capability is always enabled for all datasets.
     Enable Semantic Search – Use AI embedding models to index documents by meaning and intent. This retrieves contextually relevant.
     Enable LLM Tagging – Use LLMs to automatically apply metadata tags for data classification and semantic metadata extraction.
   • Owner and Access Control
     Owner – The dataset owner.
     Collaborators – The users who can collaborate on the dataset.
   • Selected Content
     The selected content section is dependent on the source. For example, when the source is a CTERA Portal, the content is Folders to include cloud drive folders that belong to a specific owner or without any owner.
   1. Click to add a folder.
      
   2. Click Users to expand the users and then select a user and from the list of cloud folders, select a folder.
      The folder content is displayed.
      
   3. Click Add Folder

• File Processing
  Collector – The type of collector to use.
• Schema
  The schema details including Document Summarization and Keyword Extraction. When setting Document Summarization you can add an instruction for the summarization, for example, to keep the summary short, or only one sentence, or in a different language. For Keyword Extraction you can limit the keyword extraction to specific words.
• Write Back Artifacts
  Whether write back artifacts are displayed is dependent on the source.
  Add AI-generated derivative artifacts next to the original files in the file system (creating .meta/ sidecar directories).
• Actions
  Incremental or full reindexing. Only set Full Indexing when editing a dataset and you want to reingest the whole dataset.

7. Click Save Changes.

You can follow the content ingestion for the new dataset as well as monitor the number of files and size increasing in the Files Info column.

To edit a dataset:

1. Click Datasets in the navigation pane.
   
2. Click the dataset to edit.
3. Make the changes as required.
4. Under Actions set Full Indexing when you want to reingest the whole dataset.
5. Click Save Changes.