Microsoft Internet Security and Acceleration Server 2000 Application Layer
Filtering Kit
Chapter 3
Prevent Attacks Against Microsoft Exchange with ISA Server 2000 RPC Filters

Dr. Thomas W Shinder
December
2003
Table of Contents
The
Solution: ISA Server 2000 Intelligent RPC Filter
Placing
the ISA Server 2004 Firewall on Your Network
ISA
Server 2000 Front-end Firewall Topology
ISA
Server 2000 Back-end Firewall Topology.
ISA
Server 2000 Front-end and Back-end Firewalls
ISA
Server 2000 Application Layer Filtering Web Proxy in the Perimeter Network
Secure
Exchange RPC Publishing Scenario – ISA Server 2000 as front-end firewall
Secure
Exchange RPC Publishing Scenario – ISA Server 2000 back to back firewalls
Secure
Exchange RPC Publishing Scenario – ISA Server 2000 as back-end firewall
Outlook
MAPI Clients Behind NAT Routers/Firewalls/ISA Servers
Configuring
Secure Remote Access to Microsoft Exchange RPC Services
Configuring
Secure Outbound Access to Microsoft Exchange RPC Services
Forcing
a Secure Connection from the Outlook MAPI Client
Configuring
DNS Support for Secure Exchange RPC Inbound and Outbound Access
Users benefit from working with the same e-mail client application regardless of location. Users who must switch between e-mail applications generate greater numbers of support calls, reduced productivity and lower satisfaction with their e-mail experience. The full Outlook MAPI client provides the richest e-mail experience and users typically work with this client when connected to the corporate network.
Remote access for the full Outlook MAPI client to Exchange RPC services has been problematic because of the large number of static packet filters required on traditional packet filtering firewalls. Remote access to Exchange RPC services has been made even more problematic with recent worm attacks targeted against Microsoft RPC services.
The solution to both the static packet filter and worm problems is ISA Server 2000. ISA Server 2000 firewalls provide secure remote access for the full Outlook MAPI client to Exchange RPC services. The ISA Server 2000 firewall’s intelligent RPC filter protects against attacks aimed at the Exchange Server’s RPC services and allows only valid inbound RPC connections from Outlook MAPI clients. In addition, the intelligent RPC application layer filter prevents RPC exploits from leaving the corporate network. This enables the ISA Server 2000 firewall to protect remote networks from infected clients that may exist on the corporate network. Finally, the intelligent Exchange RPC filter can be configured to force Outlook MAPI clients to use a secure RPC connection.
This document includes discussions on the problems of configuring secure remote access for Outlook MAPI clients to Exchange RPC services on the corporate network and how to solve those problems using intelligent application layer filtering ISA Server 2000 firewalls. Detailed step by step procedures are provided at the end of this document article that provide instruction on how to configure each component of a secure Outlook MAPI client remote access solution using ISA Server 2000.
A very popular method of accessing Microsoft Exchange Servers from remote sites is by using the full Outlook MAPI client. Users enjoy a better e-mail experience when using the same full Outlook 2000/2002/2003 Outlook MAPI client they use when directly connected to the corporate network. Reduced support calls, improved user productivity and increased user satisfaction are all benefits of using the same full Outlook client in the office and on the road.
The challenge for the firewall and security administrator is how to make full Outlook MAPI client remote access connections secure. Remote access to Microsoft Exchange RPC services (which is required for Outlook MAPI client access) can require a large number of statically open ports on the Internet edge firewall. The number of statically opened ports required to allow remote access to Exchange RPC services has been a barrier to enabling users an improved Outlook mail experience from remote locations.
Figure A shows how a conventional packet filtering firewall is configured to permit Outlook MAPI clients access to the Exchange RPC services. Required packet filters include:
The large number of statically opened ports on the traditional packet filtering firewall made security and firewall administrators hesitant about allowing remote access for the full Outlook MAPI client.
Figure A

Note:
You can control static port assignments on Microsoft Exchange Servers for the dynamically assigned ports. Please refer to Microsoft KB article http://support.microsoft.com/default.aspx?scid=kb;en-us;148732 for more details on this configuration.
An even more serious concern than the static port issue is the onslaught of viruses and worms designed to attack Microsoft RPC and DCOM services. The series of MSBLAST based worms (http://www.microsoft.com/security/incident/blast.asp) spread via TCP port 135. This port number must be open to allow the RPC endpoint mapper. However, if your organization uses a conventional packet filtering firewall that is not RPC application layer aware, RPC worms can attack the network via this port number. Such an attack could infect the Exchange Server and subsequently infect other machines on the corporate network.
Communications between remote e-mail clients and the Exchange Server should be secured. Outlook Web Access, SMTP, POP3, and IMAP connections to the Microsoft Exchange Servers can be secured using SSL/TLS. Outlook MAPI clients cannot use SSL to encrypt communications between client and server (with the exception of Outlook 2003/Exchange 2003 RPC over HTTP connections).
However, you can use 56-bit MD5 encryption to protect RPC communications between Outlook and Exchange. The problem here is that encryption enforcement is a client side setting. It is poor security practice to depend on users to determine the level of security on a link. Whenever possible, the level of security for a connection is forced on the server side. Exchange Server does not allow you to force exclusively secure connections to its RPC services from remote Outlook MAPI clients.
The ISA Server 2000 firewall is an intelligent application aware stateful inspection firewall. ISA Server 2000 includes an intelligent RPC application layer filter that protects against attacks by RPC worms. The intelligent RPC application layer filter also enables the ISA Server 2000 firewall administrator to force secure Outlook MAPI connections with the corporate Exchange Server. Finally, the intelligent RPC filter blocks outbound RPC worm connections from the corporate network. ISA Server 2000 advanced application layer filtering prevents RPC worm connections from leaving the corporate network and prevents hosts on your network from infecting computers on the Internet.
Figure B shows the initial RPC endpoint mapper connection between the Outlook MAPI client and the ISA Server 2000 firewall. The sequence of events:
Figure B

Figure C shows the communications sequence between the Outlook MAPI client and the Exchange Server after the endpoint mapper connection is established:
Figure C

The RPC filter can also be used to enforce secure RPC connections from Outlook MAPI clients. When this feature is enabled, connection requests from remote Outlook MAPI clients must be done via a secure encrypted channel. If the connection is not secured, the ISA Server 2000 firewall drops the client request. This allows the firewall to control the level of security for required [DS1] instead of users fixing the level of security,.
The ISA Server 2000 firewall can be the only firewall on your network, or you can integrate ISA Server 2000’s powerful application layer filtering protection with your existing firewall infrastructure. Some of the common ISA Server 2000 network topologies are:
Smaller organizations that do not already have a large investment in a current firewall infrastructure may prefer to make the ISA Server 2000 firewall a front-end firewall. The front-end firewall has a network interface on the corporate network and a network interface directly connected to the Internet. All communications into and out of the corporate network are exposed to ISA Server 2000’s deep application layer inspection.
Advantages of this configuration include:
Figure D shows the network topology for the ISA Server 2000 front-end firewall placement.
Figure D

Organizations with an existing firewall infrastructure may prefer to leave current firewalls in place and put an ISA Server 2000 firewall behind the current front-end firewalls. This topology allows third party firewalls to provide high-speed packet filtering (stateful filtering) before forwarding packets to the ISA Server 2000 intelligent firewall application filters. The perimeter network between the front and back end firewalls contains servers for publicly accessible services.
The third-party packet filtering firewalls have an interface directly connected to the Internet and an interface connected to a perimeter network between the third-party packet filtering firewalls and the ISA Server 2000 application layer aware firewall. The ISA Server 2000 firewall has an interface on the perimeter network and an interface on the protected corporate LAN.
Advantages of this configuration include:
Figure E shows the topology of the ISA Server 2000 back-end firewall topology.

The ISA Server 2000 front-end and back-end firewall configuration uses two ISA Server 2000 computers, one as the Internet edge firewall and one as the corporate LAN edge firewall. The front-end ISA Server 2000 firewall has an interface directly connected to the Internet and an interface on the perimeter network between the firewalls. The back-end ISA Server 2000 firewall has an interface on the perimeter network and an interface on the protected corporate LAN.
The advantages of this configuration include:
Figure F shows the topology for the ISA Server 2000 front-end back-end firewall configuration.

Some organizations have an existing firewall infrastructure, including both front-end and back-end firewalls. These organizations have a large investment in their current firewall infrastructures and prefer to leave them intact. You can still leverage ISA Server 2000’s application layer filtering features by making it an application layer filtering proxy. This ISA Server 2000 proxy can be placed on the perimeter network between front-end and back-end third party packet filtering firewalls. You can also place the ISA Server 2000 application layer proxy on the corporate network.
Advantages of the application layer filtering proxy configuration include:
Figure G shows the topology of the application layer filtering proxy configuration.

There are three scenarios specific to secure Exchange RPC publishing that do not require static packet filters on a front end firewall and one that does require static packet filters.
The first scenario uses the ISA Server 2000 firewall as the front end firewall. This provides a very high level of protection for the corporate network and obviates the need for static packet filters. A simple secure Exchange RPC Server Publishing Rule is the only requirement. The secure Exchange RPC filter listens for incoming connections on TCP port 135 and dynamically manages ephemeral ports required for the secure Outlook MAPI client connections.
Figure H shows the ISA Server 2000 front-end firewall topology.
Figure H

The second secure scenario involves using ISA Server 2000 as the front-end and back-end firewalls. In this scenario, the front-end ISA Server 2000 firewall publishes the external IP address on the back-end ISA Server 2000 firewall. The front-end ISA Server 2000 firewall provides a high level of protection both for servers placed on the perimeter network between the ISA Server 2000 firewalls and for machines located on the corporate network. The back-end ISA Server 2000 firewall publishes the IP address of the Exchange Server on the corporate network. No static packet filters are required and only TCP port 135 is allowed inbound on both the front-end and back-end ISA Server 2000 firewalls.
Figure I shows the topology for the front-end and back-end ISA Server 2000 secure RPC Server Publishing scenario.
Figure I

Some organizations with an existing firewall infrastructure may wish to take advantage of the rapid packet-passing features of their current firewall solutions, but still want to provide the very high level of protection a front-end ISA Server 2000 firewall provides. The solution to this problem is to use a mixed firewall infrastructure. The existing packet filtering firewall provides very fast hardware-based packet transmission (because it cannot perform stateful inspection), and the intelligent ISA Server 2000 firewall provides crucial application aware security for the published Exchange Server via its secure Exchange RPC application filter.
The packet filtering firewall can be configured to receive inbound communications for machines on the perimeter network that have been subjected to an intensive host-based hardening program. The packet filtering firewall is not able to examine exploits contained within the application layer, so host-based system hardening is the primary method of defense for these servers. Inbound connections to secure Exchange RPC and other Microsoft services (OWA, IIS, SharePoint, SMTP, POP3, DNS) can be secured by the ISA Server 2000 firewall.
Figure J shows the topology for the ISA Server 2000 and packet filtering firewall front-end firewalls and ISA Server 2000 back-end firewall scenario.
Figure J

Many organizations already have an existing front-end firewall solution in place and wish to keep the current front-end firewall infrastructure. These companies can place an intelligent ISA Server 2000 application layer filtering firewall behind the current front-end firewall infrastructure. This configuration takes advantage of reduced overhead by avoiding a reconfiguration of the current firewall infrastructure and it also takes advantage of the advanced firewall protection ISA Server 2000 can provide the corporate network.
The front-end firewall will need a number of static packet filters to support the secure Exchange RPC Server Publishing Rule. Special care is required when configuring the static packet filters to prevent servers located on the perimeter network from being compromised. The corporate network located behind the ISA Server 2000 firewall is protected by the secure Exchange RPC filter (and other application filters) and therefore is at reduced risk of attack.
Figure J shows the packet filtering front-end firewall ISA Server 2000 back-end firewall topology.
Figure J

If the Outlook client is located behind a NAT router or NAT-based conventional firewall, it will not be able to receive new mail notification requests. These new mail notification requests are not associated with the existing RPC session. Because these new mail notification messages are seen as unsolicited inbound requests, the NAT router or firewall drops the packet.
However, if the Outlook MAPI client is behind a sophisticated layer 7 aware firewall like ISA Server 2000, then Outlook will be able to receive immediate notification when new messages arrive. Outlook MAPI clients located behind an ISA Server 2000 firewall with ISA Server 2000 Feature Pack 1 installed have no problems receiving new mail notification messages.
The Outlook MAPI client is able to leverage a new and improved RPC Protocol Definition which ties into the new RPC application filter. This filter intelligently manages the connection. Intelligent connection management allows new mail notifications to flow from the Exchange Server to the remote Outlook MAPI client located behind an ISA Server 2000 firewall.
This doesn’t mean you can’t get new mail when the Outlook MAPI client is located behind a NAT router conventional NAT firewall. If you send mail to the Exchange Server, a new mail notification message can be sent through the active RPC channel. However, if there is an error in any of the RPC packets carrying the new mail notification, the notification message will not reach the Outlook MAPI client located behind a NAT router or conventional firewall. You can trigger new mail notifications by clicking on any Outlook file or folder or pressing the Send/Receive button in Outlook 2000 or you can configure the Outlook 2002 client to carry out an automatic send/receive every few minutes in the background.
Everything else (other than new mail notification messages) works fine when the Outlook client is behind the NAT router or conventional NAT firewall. If you use the Windows 2000/2003 RRAS NAT service, no further configuration is required for the NAT Routing Protocol because the Windows RRAS NAT includes an RPC proxy. If there is a NAT router in front of the Outlook MAPI client, then an RPC NAT editor is required. Most NAT routers have an RPC NAT editor included with them, but it is best to confirm this by checking the NAT router or NAT firewall documentation.
If a conventional firewall is in front of the Outlook MAPI client, the firewall administrator needs to configure it to allow a primary connection on TCP 135, and then secondary connections inbound and outbound to and from all ephemeral ports. These secondary connections are part of the same “connection bundle”, and are part of the application layer session established by the primary connection.

The Exchange RPC Server Publishing Rule uses a Protocol Definition provided by the RPC Application filter. If you disable the Application Filter, you lose the Protocol Definition, so it is important to insure that the filter is enabled. You can check the status of the RPC Filter in the Application Filters node just under the Extensions node in the left pane of the ISA Management console.
Perform the following steps to create a secure Outlook MAPI client access Server Publishing Rule:
Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

The rule will take effect soon after you click Finish. If you want the rule to apply right away, restart the Firewall service.
You can create a simple Protocol Rule allowing Outlook MAPI clients outbound access through the ISA Server 2000 firewall. This rule provides outbound access to internal network Outlook clients to connect to Exchange Servers located on the Internet. Perform the following steps to create the Protocol Rule:
1. Open the ISA Management console, expand the Servers and Arrays node and expand the server name (figure 7). Expand the Access Policy node and right click on the Protocols node. Point to New and click on Rule.
Figure 7

2. Type in a name for the Protocol Rule in the Protocol rule name text box on the Welcome to the New Protocol Wizard page (figure 8). Click Next.
Figure 8

3. Select the Allow option on the Rule Action page (figure 9). Click Next.
Figure 9

4. On the Protocols page (figure 10), select the Selected protocols option in the Apply this rule to drop down list. Select the RPC protocol from the list of Protocols. Click Next.
Figure 10

5. Select a schedule from the Use this schedule drop down list. In this example, we’ll use the Always schedule (figure 11). Click Next.
Figure 11

6. On the Client Type page (figure 12), select the Any request option and click Next.
Figure 12

7. Review your settings on the Completing the New Protocol Rule Wizard page and click Finish (figure 13).
Figure 13

The new Protocol Rule will appear in the right pane of the console.
You can configure the ISA Server 2000 firewall to require secure Outlook MAPI client RPC connections. This requires a single registry edit on the ISA Server 2000 firewall computer. Perform the following steps on the ISA Server 2000 firewall to enforce encrypted connections from Outlook MAPI clients:
1. Click Start, and then click Run. Type Regedit, and then click OK.
2. Navigate to HKEY_LOCAL_MACHINE\Software\Microsoft\FPC\PluginRPC.key. Right click on the MinimumAuthenticationLevel value in the right pane and click Modify (figure 14).
Figure 14

3. In the Edit DWORD Value dialog box, Change the value of MinimumAuthenticationLevel from 1 to 6. Click OK (figure 15).
Figure 15

4. The new value for the MinimumAuthenticationLevel value appears in the right pane of the console (figure 16)
Figure 16

The DNS infrastructure must be designed to support Outlook MAPI clients. The chief advantage of secure Exchange RPC Server Publishing is that it enables Outlook to work anywhere without requiring client reconfiguration. Outlook should correctly resolve the name of the Exchange Server regardless of location. The user should never have to reconfigure the client settings. Outlook should “just work” wherever the client system is located.
The ideal DNS configuration is the “split DNS”. The split DNS infrastructure requires two separate zones for the same domain. One of these zones supports internal network clients and the other zone supports external network clients. These two zones service the same domain or domains. The difference is that the resource record on the internal DNS zone has the private IP address of the Exchange Server and the external DNS zone has the public IP address that remote users access to connect to your published Exchange Server.
For example, if the server name is exchange.domain.com on the internal network, then you need to assure exchange.domain.com is accessible to external and internal network hosts. You can accomplish this with a split DNS configuration by creating a Host (A) record on your external DNS server for exchange.domain.com mapping to the external interface address on the ISA Server 2000 firewall you’re using in the Exchange RPC publishing rule.

If your organization does not use the same domain name for resources accessible both internally and externally, you can still access the Exchange Server via the RPC publishing rule by using local host name resolution. This requires a HOSTS file entry with the NetBIOS name of the Exchange Server computer (sometimes referred to as the “computer name”) on each Outlook client. You do not need to include the FQDN of the Exchange Server in the HOSTS file; just the NetBIOS name.
Note:
The host name portion (the leftmost label) of the FQDN must be the same as the computer name of the Exchange Server being published by the Exchange RPC Server Publishing Rule.
Configure the Outlook MAPI client to use the computer name of the Exchange
Server. It is critical that the client
be able to fully qualify the name correctly. The Outlook client computer must
use a primary domain name, or an adapter specific domain name allowing the
client to correctly fully qualify the NetBIOS name for the Exchange Server.
Outlook 2000 clients will need to be able to resolve the name of their Global
Catalog server to the same IP address that is used to publish the Exchange RPC
server
Users benefit from using the same e-mail client application regardless of their location. Users who much switch between e-mail applications generate greater numbers of support calls, reduced productivity and lower satisfaction with their e-mail experience. The full Outlook MAPI client provides the richest e-mail experience and users typically use this client when connected to the corporate network. Remote access for the full Outlook MAPI client to Exchange RPC services has been problematic because of the large number of static packet filters required on traditional packet filtering firewalls. Recent worm attacks against Microsoft RPC services have made remote access to Exchange RPC services even more problematic. ISA Server 2000 firewalls provide secure remote access for the full Outlook MAPI client to Exchange RPC services via the advantages of application layer filtering. ISA Server 2000’s intelligent RPC filter protects against attacks aimed at the Exchange Server’s RPC services and passes only valid inbound RPC connections from Outlook MAPI clients. In addition, the intelligent RPC application layer filter prevents RPC exploits from leaving the corporate network and therefore protects remote networks from infected clients that may exist on the corporate network. Finally, the intelligent Exchange RPC filter can be configured to force Outlook MAPI clients to use a secure RPC connection
[DS1]Is there a word missing here or should “for” be deleted?