Those who have worked with load balancers for applications will know that it can be a pain to troubleshoot issue where the source IP address is required because from the application’s perspective, all incoming connections have the originating IP address as the load balancer. With Citrix ADC / NetScalers, there are several methods in achieving this such as using the X-Forwarded-For header to include the source client IP address (this only works with HTTP and SSL services) or configuring direct server return (DSR) mode to allow the server to respond to clients directly by using a return path that does not flow through the Citrix ADC appliance. There are advantages and disadvantages for each method but for the purpose of this post, I will demonstrate how to configure Exchange Server 2019 (or any IIS application) to receive the source client IP with the X-Forwarded-For header.
The Scenario
Let’s assume that you have a user who is continuously locked out of their account and you have identified the event to take place on an on-premise Exchange server as you can see event ID 4625 Audit Failure events in the Security log as shown in the screenshot below:
The Exchange server is placed behind Citrix ADC / NetScalers and therefore have the IP address 172.16.5.90 of the load balancer for the Source Network Address field in the event.
Proceeding to navigate into the IIS logs on the Exchange server in the W3SVC1 folder located in the C:inetpublogsLogFiles directory:
… and opening the logs show only the source IP of the Citrix ADC / NetScaler:
Configuration IIS on Exchange Server to log the X-Forwarded-For request header
The first step to log the source IP address is to configure IIS on the Exchange server to log the X-Forwarded-For request header that is passed from the Citrix ADC / NetScaler load balancer. The following TechNet Blog does a fantastic job of demonstrating the process:
How to use X-Forwarded-For header to log actual client IP address?
https://techcommunity.microsoft.com/t5/iis-support-blog/how-to-use-x-forwarded-for-header-to-log-actual-client-ip/ba-p/873115
Below is a demonstration with an Exchange 2019 server on Windows Server 2019 and IIS version 10.0.17763.1:
Begin by launching Internet Information (IIS) Manager, navigate to either the Server node or one of the websites and then open on Logging:
We will add the X-Forwarded-For field by clicking on Select Fields beside the W3C Format dropdown menu:
Proceed to click on Add Field and add the X-Forwarded-For text as the Field Name and Source, with the Source Type as Request Header:
Note how the X-Forwarded-For is added as a Custom Field:
Apply the changes:
**Note that configuring the above on one site automatically applies it to the other sites.
Now navigate back to the IIS log files and open the latest log file and confirm that the X-Forwarded-For field is added as a header:
The following is a side by side comparison where the log on the top has the X-Forwarded-For custom field added and the bottom does not:
Configure the Citrix ADC / NetScaler to forward client source IP as X-Forwarded-For
With the IIS server configured to receive the custom X-Forwarded-For field, proceed to log into the Citrix ADC / NetScaler, navigate to Traffic Management > Load Balancing > Service Groups or Services:
For the purpose of this example, we will be configuring all of the Exchange service groups to forward the client source IP address as X-Forwarded-For (owa, activesync, rpc, ews, Autodiscover, oab, mapi, ecp).
Open the properties of the load balancing service group or service, navigate to the Settings area and click on the edit icon:
Enable the Insert Client IP Header and type in the X-Forwarded-For string for the Header text box:
Click OK to save the settings and proceed to save the settings by clicking Done.
Repeat for the rest of the Load Balancing Service Group by using the GUI or the CLI command:
set service <name> -CIP <Value> <cipHeader>
Here are the commands for each Exchange service:
set serviceGroup SVG_EX2019_owa -cip enabled X-Forwarded-For
set serviceGroup SVG_EX2019_activesync -cip enabled X-Forwarded-For
set serviceGroup SVG_EX2019_rpc -cip enabled X-Forwarded-For
set serviceGroup SVG_EX2019_ews -cip enabled X-Forwarded-For
set serviceGroup SVG_EX2019_autodiscover -cip enabled X-Forwarded-For
set serviceGroup SVG_EX2019_oab -cip enabled X-Forwarded-For
set serviceGroup SVG_EX2019_mapi -cip enabled X-Forwarded-For
set serviceGroup SVG_EX2019_ecp -cip enabled X-Forwarded-For
Testing the configuration by verifying source IP address in IIS Logs
Switching back to the Exchange Server and navigating to the IIS logs should now have the latest log reveal a value for the X-Forwarded-For field. Below is a screenshot of the log before the configuration of the Citrix ADC / NetScaler:
Below is a screenshot after the change with an IP address added to the end of each connection with the source IP:
Hope this helps anyone looking for a way to log the originating source IP address of client requests on IIS that is load balanced by a Citrix ADC / NetScaler.
One Response
Hey Terence, i've been tryin to go through this same scenario. but over port 25 SMTP. cannot seem to get that to work UNLESS we add the load balance subnet IP into our SMTP relay…
so we're load balancing our exchange servers, things seem ok, except SMTP servers dont seem to work, again unless we add the subnet IP to the SMTP relay, which we dont want to do.