Activity Search
Search agent executions using natural language queries powered by AI semantic search.
Overview
Activity Search transforms how you find and analyze agent executions. Instead of manually filtering through logs, simply describe what you're looking for in plain English. The system uses Azure AI Search with semantic ranking to find relevant executions based on meaning, not just keywords.
Examples of natural language queries:
- "What happened with Acme Corp last week?"
- "Show me all escalations related to billing"
- "Which agents processed invoices yesterday?"
- "Find errors related to email sending"
Prerequisites
Activity Search requires the Activity Log GKI (Grounded Knowledge Index) to be enabled. This feature:
- Automatically generates AI summaries of each agent execution
- Indexes summaries for semantic search
- Extracts entities (companies, people, topics) for filtering
- Manages retention based on your subscription tier
Availability by Tier
| Tier | Available | Search History Retention |
|---|---|---|
| Free | Yes | 7 days |
| Starter | Yes | 30 days |
| Pro | Yes | 90 days |
| Pro Plus | Yes | 1 year |
Enabling Activity Search
Step 1: Access the Setup Wizard
Navigate to Monitor > Activity > Activity Search. If Activity Search is not yet enabled, you'll see a setup prompt.
Click Enable Activity Search to launch the setup wizard.
Step 2: Review Introduction
The wizard explains:
- What Activity Search provides
- Your tier's retention limit
- How AI summaries are generated
Step 3: Configure Retention
Choose how long to retain searchable execution summaries:
- Default: Your tier's maximum retention
- Custom: Any value up to your tier limit
Set retention to match your compliance requirements. You can always increase retention later if you upgrade your tier.
Step 4: Provision Resources
The system automatically:
- Creates an Azure AI Search index for your tenant
- Configures semantic search settings
- Enables automatic summary generation
This typically takes 30-60 seconds.
Step 5: Optional Backfill
After provisioning, you can optionally backfill historical executions:
- Last 7 days (recommended for quick start)
- Last 30 days (if you need recent history)
- Custom date range (specify exact dates)
Backfill runs in the background and may take several minutes depending on the number of executions.
Using Activity Search
Accessing the Search Page
Navigate to Monitor > Activity > Activity Search to open the search interface.
Search Coverage Banner
When the Activity Log GKI was provisioned, Outermind recorded the cutover date - the point from which executions are indexed for semantic search. If a cutover date exists for your tenant, a blue informational banner appears at the top of the search results area:
Search covers executions from [date] forward.
This banner is informational only and cannot be dismissed. It clarifies that executions logged before the cutover date are not included in Activity Search results - they remain visible in the standard Agent Executions log, but not in semantic search.
Why you might see no banner: If Activity GKI was provisioned under an earlier version that did not record a cutover date, or provisioning has not yet run for your tenant, the banner is suppressed.
Search Input
The search box at the top accepts natural language queries. Type what you're looking for and press Enter or click the search button.
Effective search queries:
- Include specific names: "emails from John Smith"
- Mention topics: "invoices and billing issues"
- Reference time periods: "yesterday", "last week" (these work best with date filters)
- Describe outcomes: "failed executions", "escalations"
Understanding Search Results
Each result shows:
| Field | Description |
|---|---|
| Summary | AI-generated natural language description of what happened |
| Agent Name | Which agent processed this execution |
| Status | Outcome (Completed, Failed, Escalated) |
| Time | When the execution occurred |
| Duration | How long it took |
| Cost | Token costs for this execution |
| Entities | Companies, people, and topics identified |
| Highlights | Search terms highlighted in context |
Click any result to view the full Execution Details.
Filtering Results
Use the filter sidebar to narrow results:
Date Range
- Presets: Today, Yesterday, Last 7 Days, Last 30 Days
- Custom: Select specific start and end dates
Agent
Select a specific agent or view all agents.
Status
- Completed: Successful executions
- Failed: Executions that encountered errors
- Escalated: Executions that required human input
Source Type
- Agents: Standard email agent executions
- Scans: Mailbox scan operations
- AICOS: AI Chief of Staff executions
- Integrations: Third-party integration activity
Topics
Filter by automatically extracted topics like:
- billing
- scheduling
- support
- sales
Faceted Navigation
The sidebar shows facet counts for each filter option. For example:
- Status: Completed (45), Failed (3), Escalated (2)
- Topics: billing (15), support (10), scheduling (5)
Click facet values to quickly filter results.
How AI Summaries Work
When an agent execution completes, the system:
- Aggregates Logs: Collects all log entries from the execution
- Generates Summary: Uses AI to create a natural language description
- Extracts Entities: Identifies companies, people, and key terms
- Classifies Topics: Categorizes the execution by business domain
- Indexes for Search: Uploads to Azure AI Search for semantic retrieval
Example Summary
"Sales Assistant processed invoice inquiry from john@acmecorp.com regarding Invoice #1234. Searched knowledge base for pricing policy, found matching documentation, and drafted confirmation email to finance team. Execution completed successfully in 45 seconds."
Agent Tool: search_activity
Agents with Activity Search enabled can use the search_activity tool to search their own execution history. This enables self-reflection and learning from past interactions.
Tool Capabilities
- Semantic Search: Find past executions using natural language
- Execution Detail: Retrieve full details for a specific execution
- Summary Generation: Get aggregated statistics about recent activity
Access Control
| Agent Type | Default Access |
|---|---|
| Regular Agent | Own executions only |
| AICOS | All tenant executions |
Regular agents can set include_all_agents: true to search across all tenant executions if needed.
Example Usage
An agent might use this tool to:
- Remember previous interactions with a contact
- Learn from similar past requests
- Provide context about recent activity
Best Practices
Writing Effective Queries
- Be specific: "invoices from Acme" is better than "billing"
- Include context: "failed email sends last week" combines topic, status, and time
- Use natural language: Write as if asking a colleague
Combining Search and Filters
- Start with a broad search query
- Use filters to narrow by date, agent, or status
- Review facet counts to understand result distribution
- Click through to execution details for full context
Regular Monitoring
- Daily: Search for "errors" or "escalations" to catch issues
- Weekly: Review "high cost" executions for optimization opportunities
- Monthly: Analyze trends by topic or agent
Troubleshooting
Historical Executions Not Appearing in Search
Activity Search only covers executions from the cutover date onward - the date shown in the coverage banner at the top of the results area. Executions before that date were not indexed and will not appear in semantic search results. To view older executions, use the standard Agent Executions log.
No Results Found
- Check that Activity Search is enabled
- Verify the date range is on or after the cutover date shown in the banner
- Try broader search terms
- Ensure executions have completed (in-progress won't appear)
Backfill Not Completing
- Check the backfill status on the Activity Search settings page
- Large backfills may take several hours
- Failed backfills show error details in the status
Unexpected Results
- Semantic search finds related content, not just exact matches
- Use filters to narrow results after searching
- Try rephrasing your query with different terms
Data Retention
Activity Search data follows your tier's retention policy:
- Summaries older than retention limit are automatically deleted
- Deletion runs daily at 2:00 AM UTC
- Original execution logs are unaffected (managed separately)
To change retention:
- Navigate to Monitor > Activity > Activity Search
- Click Settings
- Adjust retention days (up to tier maximum)