Problem
You’ve just completed integrating Skype for Business Server 2015 with Exchange Server 2016 Unified Messaging to provide voicemail services but noticed that voicemails left for users are not being delivered and the following errors are being logged on the Exchange 2016 server:
Log Name: Application
Source: MSExchange Unified Messaging
Event ID: 1423
Level: Error
The Microsoft Exchange Unified Messaging service on the Mailbox server encountered an error while trying to process the message with header file “E:Program FilesMicrosoftExchange ServerV15UnifiedMessagingvoicemailc7d3e3ee-0212-4d27-99bd-f1960eca0698.txt”. Error details: “Microsoft.Exchange.UM.UMCore.SmtpSubmissionException: Submission to the Hub Transport server failed. The operation will be retried. —> Microsoft.Exchange.Net.ExSmtpClient.UnexpectedSmtpServerResponseException: Unexpected SMTP server response. Expected: 220, actual: 421, whole response: 421 4.3.2 Service not available
at Microsoft.Exchange.Net.ExSmtpClient.SmtpTalk.CheckResponse(ServerResponseInfo response, Int32 expectedCode)
at Microsoft.Exchange.Net.ExSmtpClient.SmtpTalk.Connect(String server, Int32 port, Boolean isSslPort)
at Microsoft.Exchange.Net.ExSmtpClient.SmtpClient.Submit()
at Microsoft.Exchange.UM.UMCore.SmtpSubmissionHelper.SubmitMessage(MessageItem message, String senderAddress, Guid senderOrgGuid, String recipientAddress, OutboundConversionOptions submissionConversionOptions, InternalExchangeServer smtpServer)
at Microsoft.Exchange.UM.UMCore.SmtpSubmissionHelper.SubmitMessage(MessageItem message, String senderAddress, Guid senderOrgGuid, String recipientAddress, OutboundConversionOptions submissionConversionOptions, String requestId)
— End of inner exception stack trace —
Server stack trace:
at Microsoft.Exchange.UM.UMCore.SmtpSubmissionHelper.HandleTransientSmtpFailure(Exception e, InternalExchangeServer smtpServer, String recipientAddress)
at Microsoft.Exchange.UM.UMCore.SmtpSubmissionHelper.SubmitMessage(MessageItem message, String senderAddress, Guid senderOrgGuid, String recipientAddress, OutboundConversionOptions submissionConversionOptions, String requestId)
at Microsoft.Exchange.UM.UMCore.SmtpSubmitStage.InternalDoSynchronousWork()
at System.Runtime.Remoting.Messaging.StackBuilderSink._PrivateProcessMessage(IntPtr md, Object[] args, Object server, Object[]& outArgs)
at System.Runtime.Remoting.Messaging.StackBuilderSink.AsyncProcessMessage(IMessage msg, IMessageSink replySink)
Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.EndInvokeHelper(Message reqMsg, Boolean bProxyCase)
at System.Runtime.Remoting.Proxies.RemotingProxy.Invoke(Object NotUsed, MessageData& msgData)
at Microsoft.Exchange.UM.UMCore.SynchronousPipelineStageBase.SynchronousWorkDelegate.EndInvoke(IAsyncResult result)
at Microsoft.Exchange.UM.UMCore.SynchronousPipelineStageBase.EndSynchronousWork(IAsyncResult r)”
Log Name: Application
Source: MSExchange Unified Messaging
Event ID: 1446
Level: Error
The Microsoft Exchange Unified Messaging service on the Mailbox server failed to process the message with header file “E:Program FilesMicrosoftExchange ServerV15UnifiedMessagingvoicemail1fbf0f79-adeb-4b6c-a7e2-792ec27a5351.txt” within “11” minutes. The server will continue to process and deliver the message, but the “MSExchangeUMAvailability: % of Messages Successfully Processed Over the Last Hour” performance counter will be decreased.
Solution
Similar to one of my previous posts from 7 years ago:
Voicemails left on Exchange UM does not get delivered to inbox – Event ID 1082 and 1035 logged
https://blog.terenceluk.com/2010/07/voicemails-left-on-exchange-um-does-not.html
… the issue was caused by the receiving connector on the Exchange server which hosts the Unified Messaging role (note that Exchange 2016 no longer splits roles into other servers). The following snippet from the event ID 1423 error logged displays what response the Exchange server is receiving when an attempt to deliver the voice is made:
The Microsoft Exchange Unified Messaging service on the Mailbox server encountered an error while trying to process the message with header file “E:Program FilesMicrosoftExchange ServerV15UnifiedMessagingvoicemailc7d3e3ee-0212-4d27-99bd-f1960eca0698.txt”. Error details: “Microsoft.Exchange.UM.UMCore.SmtpSubmissionException: Submission to the Hub Transport server failed. The operation will be retried. —> Microsoft.Exchange.Net.ExSmtpClient.UnexpectedSmtpServerResponseException: Unexpected SMTP server response. Expected: 220, actual: 421, whole response: 421 4.3.2 Service not available
You can verify this by logging onto one of the Exchange servers and initiate a telnet to port 25 to another as shown here:
421 4.3.2 Service not available
Connection to host lost.
C:>
If the environment you’re working in has many receive connectors configured:
… my suggestion is to not to attempt figuring out which routing connector may be receiving the request and rejecting the connection but rather create new ones specifically for UM with the the scope configured with the Exchange server IP address and the following security configuration:
You should see the queued files stored in the folder:
… begin to clear out when the Exchange server begins accepting the voicemail delivery connections.
