Blazor Server App Crushes on Deployment Slot Swap

Question

Blazor Server App Crushes on Deployment Slot Swap

We have a Blazor Server app that crushes sometimes when we swap deployment slots. We use Managed Identity in our database connection string to connect to an SQL Server database in Azure.

It seems like a database connection issue. Can you please help with this?

Here is the error message:

Microsoft.Data.SqlClient.SqlException (0x80131904): Login failed for user '<token-identified principal>'.

at Microsoft.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)

at Microsoft.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, SqlCommand command, Boolean callerHasConnectionLock, Boolean asyncClose)

at Microsoft.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)

at Microsoft.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)

at Microsoft.Data.SqlClient.SqlInternalConnectionTds.CompleteLogin(Boolean enlistOK)

at Microsoft.Data.SqlClient.SqlInternalConnectionTds.AttemptOneLogin(ServerInfo serverInfo, String newPassword, SecureString newSecurePassword, TimeoutTimer timeout, Boolean withFailover)

at Microsoft.Data.SqlClient.SqlInternalConnectionTds.LoginNoFailover(ServerInfo serverInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString connectionOptions, SqlCredential credential, TimeoutTimer timeout)

at Microsoft.Data.SqlClient.SqlInternalConnectionTds.OpenLoginEnlist(TimeoutTimer timeout, SqlConnectionString connectionOptions, SqlCredential credential, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance)

at Microsoft.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentity identity, SqlConnectionString connectionOptions, SqlCredential credential, Object providerInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString userConnectionOptions, SessionData reconnectSessionData, Boolean applyTransientFaultHandling, String accessToken, DbConnectionPool pool, Func`3 accessTokenCallback)

at Microsoft.Data.SqlClient.SqlConnectionFactory.CreateConnection(DbConnectionOptions options, DbConnectionPoolKey poolKey, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningConnection, DbConnectionOptions userOptions)

at Microsoft.Data.ProviderBase.DbConnectionFactory.CreatePooledConnection(DbConnectionPool pool, DbConnection owningObject, DbConnectionOptions options, DbConnectionPoolKey poolKey, DbConnectionOptions userOptions)

at Microsoft.Data.ProviderBase.DbConnectionPool.CreateObject(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal oldConnection)

at Microsoft.Data.ProviderBase.DbConnectionPool.UserCreateRequest(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal oldConnection)

at Microsoft.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, UInt32 waitForMultipleObjectsTimeout, Boolean allowCreate, Boolean onlyOneCheckConnection, DbConnectionOptions userOptions, DbConnectionInternal& connection)

at Microsoft.Data.ProviderBase.DbConnectionPool.WaitForPendingOpen()

--- End of stack trace from previous location ---

at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.OpenInternalAsync(Boolean errorsExpected, CancellationToken cancellationToken)

at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.OpenAsync(CancellationToken cancellationToken, Boolean errorsExpected)

at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReaderAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)

at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.AsyncEnumerator.InitializeReaderAsync(AsyncEnumerator enumerator, CancellationToken cancellationToken)

at Microsoft.EntityFrameworkCore.SqlServer.Storage.Internal.SqlServerExecutionStrategy.ExecuteAsyncTState,TResult

at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.AsyncEnumerator.MoveNextAsync()

at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.SingleAsyncTSource

2 answers

Your answer

Answer 1

Pradeep M7,785Microsoft External Staff

Hi Hayk Shirinyan,

It seems the Managed Identity token may not be available or valid during the deployment slot swap. To address this, please ensure that Managed Identity is enabled for both the staging and production slots, and that the SQL Server has the necessary Azure AD permissions assigned.

We also recommend adding retry logic during application startup to handle any transient connection issues that may occur during the swap process.

If the issue continues, please share the documentation or learning path you're following, along with any relevant code snippets. This will help us provide more targeted assistance.

If you found this answer helpful, kindly click the "Accept Answer/Upvote" button.

Hayk Shirinyan0Reputation points
2025-04-15T04:15:39.1066667+00:00
Thanks for your response. I'm not following any learning path. This is a production application with the issue, and I'm a professional developer.
Managed Identity is enabled on both deployment slots and permissions are configured in the databases.
Can you please explain how we can add retry logic? There is no specific database connection that fails. Any connection may fail after the swap. It seems like there is some kind of a caching issue with Blazor Server architecture when using Managed Identity and deployment slots.
Pradeep M7,785Reputation pointsMicrosoft External Staff
2025-04-16T12:12:09.9133333+00:00
Hi Hayk Shirinyan,
Thank you for the additional context. Since Managed Identity is correctly configured on both slots, the issue likely stems from a transient authentication failure during the slot swap process. These can occur if the identity token is not immediately available or has expired.
To help mitigate this, consider enabling retry logic in your DbContext configuration using EF Core’s built-in support:
services.AddDbContext<MyDbContext>(options => options.UseSqlServer(configuration.GetConnectionString("DefaultConnection"), sqlOptions => sqlOptions.EnableRetryOnFailure(5, TimeSpan.FromSeconds(10), null)));
Additionally, because Blazor Server maintains persistent connections, it’s important to avoid caching tokens or any scoped services like DbContext. Each user interaction should receive a fresh instance through dependency injection.
As a temporary workaround, you might also consider adding a brief warm-up check after slot swaps to ensure the application is fully initialized before it begins handling live traffic.
Hayk Shirinyan0Reputation points
2025-04-19T20:46:07.1433333+00:00
Thanks, I will try that and let you know if it doesn't work.
Hayk Shirinyan0Reputation points
2025-04-27T20:53:05.34+00:00
We tried your suggestion. We added a custom warm-up endpoint and used that during deployment slot swaps, but it didn't work. What's interesting is that the warm-up endpoint can successfully connect to the database, but any code that tries to connect through Blazor fails. It's as if there is some kind of a caching issue with Blazor server architecture.
When this error happens, it's not like we get intermittent database connection errors. All connections from Blazor fail, so it doesn't seem like a retry mechanism would work. I feel like the production deployment slot uses the staging slot's managed identity for some reason to connect to database. It feels like it tries to use a token it got from staging slot to connect to production database after the swap completes.
Pradeep M7,785Reputation pointsMicrosoft External Staff
2025-04-28T05:00:37.4633333+00:00
Hi Hayk Shirinyan,
It appears that the issue may be related to token caching or the reuse of tokens between deployment slots after the swap. To address this, I recommend the following:
1.Refresh Tokens During Slot Swap: Ensure that any cached tokens are cleared or refreshed during the application startup. You may need to use libraries like Azure.Identity or MSAL to explicitly refresh the tokens.
2.Use Separate Managed Identities for Each Slot: Consider assigning separate managed identities to the staging and production slots. This will ensure that each slot uses its own identity after the swap.
3.Handle Token Expiry: Implement logic to handle token expiry and ensure that the application properly refreshes the token when needed, particularly after the deployment slot swap.
4.Investigate Token Caching in Blazor: Confirm that Blazor is not reusing the token from the staging slot. You may need to reset the DbContext or refresh services during the application startup to ensure the correct token is used.
5.Enable Diagnostic Logging: Add logging to track the token being used for database connections. This will help confirm whether the correct token is being used after the swap.

Answer 2

When a Blazor Server app crashes during a deployment slot swap, it’s often due to state or configuration mismatches between the source and target slots. Common culprits include unhandled startup exceptions, misconfigured app settings, or missing environment-specific dependencies. To debug, review the application logs in Azure App Service, especially during the swap operation. Ensure that both slots have identical configuration values and connection strings. Also, verify database migrations or external services aren’t causing conflicts during warm-up. Implementing robust error handling and using App Service's “auto-heal” and diagnostic tools can prevent crashes and offer insights to resolve deployment issues smoothly. Read It For More Information

Share via

Blazor Server App Crushes on Deployment Slot Swap

2 answers

Your answer