Skip to main content

LLM Providers

Configure and manage AI service providers that power your ExecAssist agents.

Overview​

LLM (Large Language Model) Providers are the AI services that power agent intelligence. ExecAssist supports multiple providers, allowing you to choose based on capability, cost, and compliance requirements.

Supported Providers​

ProviderModelsBest For
AnthropicClaude Sonnet, Claude Opus, Claude HaikuGeneral use, excellent reasoning
OpenAIGPT-4, GPT-4 Turbo, GPT-3.5Wide compatibility, proven track record
GoogleGemini Pro, Gemini FlashMultimodal capabilities
xAIGrokAlternative reasoning models
Azure OpenAIGPT-4, GPT-4 TurboEnterprise compliance, data residency

Adding a Provider​

Step 1: Navigate to Providers​

Go to Configuration > AI Providers. The page shows:

  • Configured Providers - Providers you've already set up with their active models
  • Add Another Provider - Available providers you can configure

Step 2: Select a Provider​

Click on any unconfigured provider card (e.g., Anthropic, OpenAI, Google). This opens the configuration view with instructions for obtaining an API key.

Step 3: Enter and Validate API Key​

  1. Follow the displayed instructions to obtain your API key from the provider
  2. Enter your API key in the field
  3. Click Validate to verify the key works

For Azure OpenAI, you'll also need to provide:

  • Endpoint URL - Your Azure OpenAI endpoint (e.g., https://your-resource.openai.azure.com)

Once validated, you'll see a green "Valid" badge next to the API key field.

Step 4: Add Models​

After API key validation, you can add models using the quick-add buttons:

  1. One-click model addition - Click any model pill button to instantly add and validate that model
  2. Custom models - Click "+ Custom Model" to add models not in the pre-defined list

Models are automatically saved when validation succeeds. You'll see:

  • Green checkmark - Model validated and saved
  • Spinning indicator - Validation in progress
  • Red error - Validation failed (click "Retry" to try again)

Step 5: Add More Providers​

Click Add another Provider to return to the provider selection view and configure additional providers.

Managing Providers​

View Configured Providers​

The AI Providers page displays each configured provider as a card showing:

  • Provider Name - The provider type (e.g., Anthropic, OpenAI)
  • Active Badge - Green badge indicating the provider is configured
  • Model List - All models added to this provider with checkmarks

Edit a Provider​

  1. Click the gear icon on any configured provider card
  2. Add or remove models as needed
  3. Changes are saved automatically when models are validated

Note: To change an API key, you'll need to delete and re-add the provider.

Remove a Model​

  1. Click the gear icon to edit the provider
  2. Click the X button next to any model to remove it
  3. If the model is in use, a confirmation dialog shows which components depend on it:
    • Agents using this model
    • SMEs using this model
    • Safety Gateway if configured with this model
  4. Review the dependencies and confirm deletion
  5. After deleting, you'll need to update the affected components to use a different model

Delete a Provider​

  1. Click the trash icon on the provider card
  2. A confirmation dialog shows if any models from this provider are in use
  3. The provider and all its models will be removed

Warning: Before deleting a provider, ensure no agents, SMEs, or the Safety Gateway are using its models. Reassign them to different models first.

Provider Status​

Healthy (Green)​

The provider is connected and responding normally.

Unhealthy (Red)​

The provider cannot be reached. Common causes:

  • Invalid or expired API key
  • Network connectivity issues
  • Provider service outage
  • Rate limiting

Checking Provider Health​

  1. Go to the providers page
  2. Click Refresh to update status
  3. Click on a provider to see detailed diagnostics

Cost Management​

Understanding Costs​

LLM costs depend on:

  • Input tokens - Text sent to the model
  • Output tokens - Text generated by the model
  • Model choice - More capable models cost more

Monitoring Usage​

View costs in Admin > Account > Usage & Billing:

  • Total spend by provider
  • Cost per execution
  • Token usage trends
  • Per-agent cost breakdown

Cost Optimization Tips​

  1. Use appropriate models - Haiku for simple tasks, Sonnet for complex
  2. Optimize prompts - Shorter, clearer instructions use fewer tokens
  3. Set token limits - Configure max tokens per response
  4. Monitor outliers - Check for unexpectedly expensive executions

Best Practices​

Multiple Providers​

Consider configuring multiple providers for:

  • Redundancy - Failover if one provider has issues
  • Cost optimization - Use cheaper providers for simple tasks
  • Compliance - Azure for data residency requirements
  • Capability matching - Different models for different use cases

Security​

  • Rotate API keys regularly
  • Use environment-specific providers (dev, staging, prod)
  • Monitor for unusual usage patterns
  • Limit access to provider management

Provider Selection​

ScenarioRecommended Provider
General useAnthropic (Claude Sonnet)
Cost-sensitiveAnthropic (Claude Haiku) or GPT-3.5
Enterprise/complianceAzure OpenAI
Complex reasoningAnthropic (Claude Opus) or GPT-4

Troubleshooting​

Invalid API Key​

Symptoms: Provider shows red status, agents fail

Solutions:

  1. Verify the API key is correct (no extra spaces)
  2. Check the key hasn't expired
  3. Ensure the key has appropriate permissions

Rate Limiting​

Symptoms: Intermittent failures, 429 errors

Solutions:

  1. Check your provider's rate limits
  2. Reduce concurrent agent executions
  3. Upgrade your provider plan if needed

Connection Timeout​

Symptoms: Provider occasionally shows unhealthy

Solutions:

  1. Check network connectivity
  2. Verify firewall rules allow outbound HTTPS
  3. Check provider status page for outages