Cleaning and Categorizing
Document Cleaning Summaries in supermemory
supermemory provides advanced configuration options to customize your content processing pipeline. At its core is an AI-powered system that can automatically analyze, categorize, and filter your content based on your specific needs.
Configuration Schema
Core Settings
shouldLLMFilter
- Type:
boolean
- Required: No (defaults to
false
) - Description: Master switch for AI-powered content analysis. Must be enabled to use any of the advanced filtering features.
categories
- Type:
string[]
- Limits: Each category must be 1-50 characters
- Required: No
- Description: Define custom categories for content classification. When specified, the AI will only use these categories. If not specified, it will generate 3-5 relevant categories automatically.
filterPrompt
- Type:
string
- Limits: 1-750 characters
- Required: No
- Description: Custom instructions for the AI on how to analyze and categorize content. Use this to guide the categorization process based on your specific needs.
includeItems & excludeItems
- Type:
string[]
- Limits: Each item must be 1-20 characters
- Required: No
- Description: Fine-tune content filtering by specifying items to explicitly include or exclude during processing.
filterTags
- Type: Array of
{ tag: string, score: number }
- Limits: Tags must be 1-50 characters, scores must be ≥ 0
- Required: No
- Description: Define weighted tags for more nuanced content classification. Useful for prioritization and advanced filtering.
Content Processing Pipeline
When content is ingested with LLM filtering enabled:
-
Initial Processing
- Content is extracted and normalized
- Basic metadata (title, description) is captured
-
AI Analysis
- Content is analyzed based on your
filterPrompt
- Categories are assigned (either from your predefined list or auto-generated)
- Tags are evaluated and scored
- Content is analyzed based on your
-
Chunking & Indexing
- Content is split into semantic chunks
- Each chunk is embedded for efficient search
- Metadata and classifications are stored
Example Use Cases
1. Customer Feedback System
2. Content Moderation
Important: All filtering features (
categories
,filterPrompt
,includeItems
,excludeItems
,filterTags
) requireshouldLLMFilter
to be enabled. Attempting to use these features without enablingshouldLLMFilter
will result in a 400 error.