I recently had to rebuild a failed Exchange 2010 server that was a part of a DAG and while the rebuild went quite smoothly, I had quite a few issues reseeding the database back onto the new server so I thought it would be good to blog 2 of the main issues I had:
Problem #1
Attempting to add a mailbox database copy with the cmdlet Add-MailboxDatabaseCopy throws the error:
[PS] C:Windowssystem32>Add-MailboxDatabaseCopy -Identity “EXDB-02” -MailboxServer MBX-01
A source-side operation failed. Error An error occurred while performing the seed operation. Error: Failed to open a lo
g truncation context to source server ‘MBX-02.someDomain.NET’. Hresult: 0xfffffae7. Error: The database was eithe
r not found or was not replicated.. [Database: EXDB-02, Server: MBX-01.someDomain.NET]
+ CategoryInfo : InvalidOperation: (:) [Add-MailboxDatabaseCopy], SeedInProgressException
+ FullyQualifiedErrorId : 89031D8B,Microsoft.Exchange.Management.SystemConfigurationTasks.AddMailboxDatabaseCopy
[PS] C:Windowssystem32>
You notice that the mailbox server has been added as one of the servers for the mailbox database but the Copy Status is listed as Suspended:
You attempt to use the Resume Database Copy option in the EMC to resume the seeding:
… but notice that the Copy Status quickly changes into Failed and Suspended:
Alternatively, if you use the EMC to add the mailbox database copy, you see the following error:
Summary: 1 item(s). 0 succeeded, 1 failed.
Elapsed time: 00:00:34
EXDB-02
Failed
Error:
A source-side operation failed. Error An error occurred while performing the seed operation. Error: An error occurred while processing a request on server ‘MBX-02’. Error: Couldn’t open backup file handle for database ‘EXDB-02’ to server ‘MBX-02’. Hresult: 0x50d. Error: A database backup is already in progress. Please verify that no other seeding or incremental reseeding operations are started for this database, and then try the operation again by rerunning the Update-MailboxDatabaseCopy cmdlet.. [Database: EXDB-02, Server: MBX-01.someDomain.NET]
An error occurred while processing a request on server ‘MBX-02’. Error: Couldn’t open backup file handle for database ‘EXDB-02’ to server ‘MBX-02’. Hresult: 0x50d. Error: A database backup is already in progress. Please verify that no other seeding or incremental reseeding operations are started for this database, and then try the operation again by rerunning the Update-MailboxDatabaseCopy cmdlet.
Couldn’t open backup file handle for database ‘EXDB-02’ to server ‘MBX-02’. Hresult: 0x50d. Error: A database backup is already in progress. Please verify that no other seeding or incremental reseeding operations are started for this database, and then try the operation again by rerunning the Update-MailboxDatabaseCopy cmdlet.
Click here for help… http://technet.microsoft.com/en-US/library/ms.exch.err.default(EXCHG.141).aspx?v=14.2.247.1&t=exchgf1&e=ms.exch.err.Ex4543D9
Exchange Management Shell command attempted:
Add-MailboxDatabaseCopy -Identity ‘EXDB-02’ -MailboxServer ‘MBX-01’ -ActivationPreference ‘2’
Elapsed Time: 00:00:34
As ridiculous as may sound, the way I got the seeding to work was to dismount all of the databases on the active server (the one that wasn’t rebuilt), restart the server, mount the databases, and started the seeding.
Problem #2
You notice that while the seeding of a database starts and seemingly completes, the Copy Status ends up being as Failed and Suspended:
… if you attempt to resume the seeding, the Copy Status will end up being Failed.
Another symptom observed is that the logs on the active database are not truncated even if you turn on circular logging.
Also, you notice the following event logged on the active server:
Exchange Search Indexer has temporarily disabled indexing of the mailbox database EXDB-01 (GUID = 52c3b6c8-c3ef-4912-8b04-e6f7e3eeb438) due to an error (Microsoft.Mapi.MapiExceptionMdbOffline: MapiExceptionMdbOffline: Unable to read events. (hr=0x80004005, ec=1142)
Diagnostic context:
Lid: 33865
Lid: 1494 —- Remote Context Beg —-
Lid: 44215
Lid: 60049 StoreEc: 0x8004010F
Lid: 49469
Lid: 65341 StoreEc: 0x8004010F
Lid: 56125
Lid: 47933 StoreEc: 0x8004010F
Lid: 32829
Lid: 49213 StoreEc: 0x8004010F
Lid: 48573
Lid: 64957 StoreEc: 0x8004010F
Lid: 20057 StoreEc: 0x476
Lid: 1750 —- Remote Context End —-
Lid: 28777 StoreEc: 0x476
Lid: 20098
Lid: 20585 StoreEc: 0x476
at Microsoft.Mapi.MapiExceptionHelper.ThrowIfError(String message, Int32 hresult, SafeExInterfaceHandle iUnknown, Exception innerException)
at Microsoft.Mapi.MapiEventManager.ReadEvents(Int64 startCounter, Int32 eventCountWanted, Int32 eventCountToCheck, Restriction filter, ReadEventsFlags flags, Boolean includeSid, Int64& endCounter)
at Microsoft.Exchange.Search.RetriableOperations.ReadEvents(ThreadLocalCrawlData unused1, MapiEventManager eventManager, Int64 watermark, Int32 eventCount, Int64& endCounter)
at Microsoft.Exchange.Search.RetriableOperations.DoRetriableMapiOperation[SourceType,ReturnType,Parameter1Type,Parameter2Type,Parameter3Type](ThreadLocalCrawlData crawlData, SourceType source, Parameter1Type parameter1, Parameter2Type parameter2, Parameter3Type& parameter3, MapiOperationDelegate`5 operationDelegate)
at Microsoft.Exchange.Search.NotificationWatcher.GetMapiEvents(Int32 maxEvents, NotificationQueue notificationQueue, Int64& endCount)
at Microsoft.Exchange.Search.NotificationWatcher.NotificationWatcherThread()), and Operations Manager would never alert it.
What ended up fixing this issue for me was to rebuild the full-text index catalog for all of my mailbox databases as per the following KB:
How to Rebuild the Full-Text Index Catalog
http://technet.microsoft.com/en-us/library/aa995966(EXCHG.80).aspx
**Note that the article states that it applies to Exchange 2007 but it works for 2010 as well.
Then proceeded to dismount and remount the database that wasn’t truncating the logs to get the logs to truncate.
Once the problematic database was back in good health, I was able to reseed the database onto the new mailbox server without running into the same issue.
