Skip to main content

Switching RAG Models

You can change the Azure OpenAI chat completion model on an active Grounded Knowledge Index without re-provisioning the index. This is useful when you want to upgrade to a newer or more capable model, or switch to a more cost-effective option.

When to Switch Models​

Common reasons for model reconfiguration:

  • Upgrade to newer models - Move from gpt-4o to gpt-4.1-mini for improved performance
  • Cost optimization - Switch to a smaller model (e.g., gpt-4.1-nano) for indexes with simpler queries
  • Performance tuning - Try a different model to improve answer quality for your specific content

Supported Models​

The following Azure OpenAI models are supported for Grounded Knowledge Indexes:

ModelNotes
gpt-4oPrevious generation, broadly available
gpt-4o-miniPrevious generation, cost-effective
gpt-4.1Current generation, high capability
gpt-4.1-miniCurrent generation, recommended balance of cost and quality
gpt-4.1-nanoCurrent generation, most cost-effective
gpt-5Latest generation, highest capability
gpt-5-miniLatest generation, balanced
gpt-5-nanoLatest generation, cost-effective
info

The model must be deployed in your Azure OpenAI resource before you can switch to it. If the deployment does not exist, the reconfiguration will fail at the verification step.

Reconfiguration Wizard​

The model reconfiguration uses a 4-step mini-wizard accessible from the index details page.

Step 1: Test New Model​

  1. Navigate to Build > Knowledge
  2. Click on the index you want to reconfigure
  3. Click the Edit button on the index details page
  4. Enter the new deployment name (the name of your Azure OpenAI model deployment)
  5. The wizard tests connectivity to the new model to verify it exists and is accessible

The wizard runs a test search query against the index using the new model to confirm that answer synthesis works correctly with the updated configuration.

Step 3: Execute Reconfiguration​

The system updates the Azure provider record and the Knowledge Base configuration to use the new model. This step:

  1. Updates the provider's deployment name
  2. Reconfigures the Knowledge Base to reference the new model
  3. Validates the updated configuration

Step 4: Confirm​

Review the reconfiguration results:

  • Verify the new model is active
  • Run a test query to confirm answer quality
  • The index continues operating without interruption

Rollback Behavior​

If the reconfiguration fails at any step after the provider record is updated, the system automatically rolls back the provider record to its previous configuration. This ensures your index remains functional even if the model switch encounters an error.

warning

While the rollback mechanism protects against partial failures, it is still recommended to verify your new model deployment is accessible before starting the reconfiguration process.

Best Practices​

  1. Deploy the model first - Ensure the target model is deployed and accessible in your Azure OpenAI resource before starting reconfiguration
  2. Test during low-traffic periods - While reconfiguration is fast, performing it during off-peak hours minimizes any brief disruption
  3. Compare answer quality - After switching, run several test queries to compare answer quality with the previous model
  4. Document the change - Note which model each index uses so your team can track configurations across indexes