Configuring a Secure Internal SMTP Relay with the SMTP Filter and Message Screener

 

An SMTP relay is a machine that accepts SMTP messages for mail domains for which is it not authoritative. The most common example of an SMTP relay is an SMTP server run by an ISP that accepts SMTP messages destined to other domains. The user logs onto his ISP and his email client software is configured to use the SMTP server at his ISP. The user sends an SMTP message to a user at domain.com, which is not run by the ISP the user connects to. The SMTP message is sent to the ISP’s SMTP server and the ISP’s SMTP server relays the mail to the appropriate SMTP server for the domain.com domain.

 

You can create an SMTP relay on your internal network that accepts incoming mail to the domains that you are responsible for. This relay can leverage the ISA Server 2000 SMTP Message Screener to block messages based on source address or domain, attachment characteristics or keywords. In addition, you can configure a second, secure authenticating SMTP virtual server on the SMTP relay computer to allow your external users to relay mail to any domain.

 

The primary advantage of using an SMTP relay for incoming connections is that you avoid allowing anonymous external computers from creating new inbound SMTP connections to your Exchange Server. The incoming connections are made to the SMTP relay and the SMTP relay forwards legitimate mail to email domains hosted on the Exchange Server to the Exchange Server machine. In addition, the SMTP relay machine absorbs the processing costs involved with the deep inspection of SMTP messages that the ISA Server 2000 SMTP filter performs.

 

You need to carry out the following procedures to create an anonymous inbound SMTP relay that sends mail to domains under your administrative control and create a secure authenticating SMTP relay your external users can use to relay mail to all mail domains:

 

 

The remainder of the ISA Server 2000 Exchange Server 2000/2003 Deployment Kit document covers each of these procedures in detail.

 

 

 

Installing Windows Server 2003 on the Firewall Computer

 

The computer that becomes the ISA Server 2000 firewall must meet the following minimum requirements:

 

 

The ISA Server firewall and Web caching components work very well on very modest hardware. This is true even when the SMTP filter is enabled and protecting the published co-located SMTP server. However, the SMTP Message Screener can be very processor intensive. This is why I recommend that you use a processor with a minimum of rating of 1.5 MHz. This is especially true if you plan on running an authenticating and non-authenticating SMTP relay on the same computer.

 

 

 

Installing ISA Server 2000 on the Firewall Computer

 

Requirements and procedures for installing ISA Server 2000 on a Windows Server 2003 are slightly different from those for installing ISA Server 2000 on a Windows 2000 computer. Please refer to ISA Server 2000 Exchange Server 2000/2003 Deployment Kit document Installing ISA Server 2000 on Windows Server 2003 for details on how to install ISA Server 2000 on a Windows Server 2003 computer.

 

 

 

Configure DCOM Permissions on the Firewall Computer

 

The SMTP relay on the internal network using DCOM to communicate with the SMTP filter components on the ISA Sever firewall. DCOM permissions must be configured to allow the SMTP relay on the internal network to communicate with the filter. Perform the following steps to configure the proper DCOM permissions:

 

  1. Click Start and then click the Run command. Type dcomcnfg.exe in the Open text box and click OK (figure 1).

 

Figure 1

 


  1. In the Component Services console (figure 2), expand the Component Services node, and then expand the Computers node. Expand the My Computer node and then click on the DCOM Config node. Right click on the VendorDataClass entry in the right pane of the console and click Properties.

 

Figure 2

 


  1. On the General tab, click the down arrow for the Authentication drop down list box and select the None option (figure 3). Click Apply.

 

Figure 3

 


  1. Click on the Location tab (figure 4). Put a checkmark in the run application on the computer where the data is located checkbox. Put a checkmark in the run application on the following computer checkbox. Then type in the name of the SMTP relay computer on the internal network. Make sure that the ISA Server firewall can resolve this name. You can use either a NetBIOS name or a fully qualified domain name. Use WINS to help resolve NetBIOS names and DNS for resolve FQDNs.

 

Figure 4

 


  1. Click on the Security tab (figure 5). Select in the Customize option in the Launch Permissions, Access Permissions and Configuration Permissions frames. Click the Edit button in the Launch Permissions frame.

 

Figure 5

 


  1. In the Launch Permissions dialog box, click the Add button (figure 6). In the Select Users, Computers, or Groups dialog box, type Everyone in the Enter the object names to select text box. Click the Check Names button to confirm that the system recognizes the entry. Click OK in the Select Users, Computers, or Group dialog box.

 

Figure 6

 


  1. Notice that the Everyone group is automatically granted the Allow permission for Launch Permission (figure 7). Click OK in the Launch Permission dialog box.

 

Figure 7

 


  1. Click on the Edit button in the Access Permissions frame (figure 8).

 

Figure 8

 


  1. Click the Add button in the Access Permissions dialog box (figure 9). In the Select users, Computers, or Groups dialog box, type Everyone in the Enter the object names to select text box. Click the Check Names button to confirm that the system recognizes the entry. Click OK in the Select Users, Computers, or Groups dialog box.

 

Figure 9

 


  1. Notice in the Access Permission dialog box (figure 10) that the Everyone group is automatically assigned the Allow permission to the Access Permission permission. Click OK.

 

Figure 10

 


  1. Click the Edit button in the Configuration Permissions frame (figure 11).

 

Figure 11

 


  1. In the Change Configuration Permission dialog box, click the Add button (figure 12). In the Select Users, Computers, or Groups dialog box, type Everyone in the Enter the object names to select text box. Click the Check Names button to confirm that the system recognizes the entry. Click OK in the Select users, Computers, or Groups dialog box.

 

Figure 12

 


  1. Notice in the Change Configuration Permission dialog box that the Everyone group is assigned the Allow permission to the Full Control and Read permissions (figure 13). Click OK.

 

Figure 13

 


  1. Click on the Identify tab (figure 14). Select the This user option. Type in a user name in the User name text box. This user must be an administrator on the local machine, or a domain administrator. Enter this user’s password and then confirm the password. Click Apply and then click OK.

 

Figure 14

 


  1. Close the Component Services console (figure 15)

 

Figure 15

 

 

Install Windows Server 2003 on the SMTP Relay Computer

 

The SMTP relay computer must meet the basic hardware requirements for installing Windows Server 2003. In addition, consider the following options for the SMTP relay machine:

 

 

In this ISA Server 2000 Exchange Server 2000/2003 Deployment Kit article, we will assume that the machine is not a member of the user domain and that you will assign the users a group account to authenticate with the secure authenticating SMTP server. If the SMTP relay were a member of the Windows domain that contained user accounts, you might consider allowing users to use their own user account to authenticate with the SMTP relay computer.

 

Install the SMTP Service on the SMTP Relay Computer

 

For security reasons, IIS 6.0 is not installed by default on a Windows Server 2003 computer. You must install the IIS 6.0 SMTP service on the Windows Server 2003 SMTP relay.

 

Perform the following steps to install the IIS 6.0 SMTP relay computer:

 

1.       Click Start, point to Control Panel and click on Add or Remove Programs. In the Add or Remove Programs window, click the Add/Remove Windows Components button. You will see a Windows Setup dialog box asking your to please wait (figure 16).

 

Figure 16

 

2.       In the Windows Components dialog box (figure 17), click on the Application Server entry (do not put a checkmark in its checkbox!). Click Details.

 

Figure 17

 

3.       In the Application Server dialog box (figure 18), Click on the Internet Information Services (IIS) entry (do not put a checkmark in its checkbox!). Click Details.

 

Figure 18 (fig18)

 

4.       In the Internet Information Services (IIS) dialog box, put a checkmark in the SMTP Service checkbox (figure 19). Click OK.

 

Figure 19

 

5.       Click OK in the Application Server dialog box (figure 20).

 

Figure 20

 

6.       Click Next on the Windows Components dialog box (figure 21).

 

Figure 21

 

7.       A progress bar appears as the application installs (figure 22).

 

Figure 22

 

8.       Click Finish on the Completing the Windows Components Wizard page (figure 23).

 

Figure 23

 

 

 

 

Configure the First Virtual SMTP Server (Non-Authenticating) for Anonymous Inbound Relay

 

We will run both an anonymous inbound SMTP relay that allows Internet SMTP servers to relay SMTP mail to the domains under your administrative control and a secure authenticating SMTP relay that you external users can use to relay mail to all domains, internal and external. We will configure the Default SMTP Virtual Service to provide the anonymous inbound relay. Later we will install a second virtual SMTP server that you will use as the secure authenticating SMTP relay.

 

1.       Click Start and point to Administrative Tools. Click on the Internet Information Services (IIS) Manager entry (figure 24).

 

Figure 24

 

2.       Right click on the Default SMTP Virtual Server entry in the left pane of the console (figure 25). Click the Properties command.

 

Figure 25

 

3.       In the Default SMTP Virtual Server Properties dialog box (figure 26), click the General tab. Click the down arrow for the IP address drop down list box and select the IP address that you want the anonymous inbound SMTP relay to use. This anonymous relay accepts mail for the mail domains that are under your administrative control and drops all other inbound mail. Internet SMTP servers use this machine to send mail to the Exchange Server. Click Apply after selecting the IP address.

 

Figure 26

 

 

4.       Click on the Messages tab (figure 27). You have the following options:

 

Limit message size to (KB)

This is what Microsoft SMTP Service will advertise, in kilobytes, as the maximum message size this SMTP virtual server will accept. If a mail client sends a message that exceeds the limit, the client will get an error. If a remote server supports EHLO, it will detect the advertised maximum message size value when it connects to the SMTP virtual server and won't even attempt to deliver a message that exceeds the limit. Instead it will simply NDR the sender of the message. A remote server that doesn't support EHLO will try to send a message that exceeds the size limit, but will still end up sending an NDR to the sender when the message doesn't go through. The default is 2048 KB. The minimum value is 1KB. To have no limit, clear the check box.

 

Limit session size to (KB)

This is the maximum amount of data, in kilobytes, accepted during the total connection. It is the sum of all messages sent during the connection (applying to the message body only). Type a value larger than the Limit message size to (KB). This maximum should be set carefully, because the connecting message transfer agent (MTA) is likely to resubmit the message repeatedly. The default size is 10240 KB. This value should be greater than or equal to the value entered for Limit message size to (KB). To have no limit, clear the check box.

 

Limit number of message per connection to

When the check box is selected, this option enables you to limit the number of messages sent in a single connection. The default is 20. This feature also provides a method to increase system performance by using multiple connections to deliver messages to a remote domain. Consequently, once the set limit is reached, a new connection is automatically opened and the transmission continues until all messages are delivered. To disable this feature and have no limit, clear the check box.

 

Limit number of recipients per message to

This setting limits the maximum number of recipients for a single message. The default is 100, which is the minimum required number specified in Request for Comments (RFC) 821. To disable this feature and have no limit, clear the check box. Some clients return messages with a non-delivery report (NDR) once an error message is received indicating that the maximum number of recipients has been exceeded. A server running Microsoft SMTP Service does not return messages with an NDR in this instance. It opens a new connection immediately and processes the remaining recipients. For example, if the recipient limit is set to 100 and a message with 105 recipients is being transmitted, the first 100 are delivered in one connection after receipt of the error message. Then a new connection is opened and the message is processed for the remaining five recipients.

 

Send copy of Non-Delivery Report to

When a message is undeliverable, it is returned to the sender with a non-delivery report (NDR). You can designate that copies of the NDR are sent to a specific SMTP mailbox. Type an e-mail address for the mailbox.

 

Badmail Directory

When a message is undeliverable, it is returned to the sender with a non-delivery report (NDR). You can designate that copies of the NDR are sent to a location of your choice.

All NDRs go through the same delivery process as other messages, including attempts to resend the message. If the NDR has reached the retry limit and cannot be delivered to the sender, a copy of the message is placed in the Badmail directory. Messages placed in the Badmail directory cannot be delivered or returned. Check the directory regularly and reconcile the messages, because a full directory may adversely affect Microsoft SMTP Service performance.

 

If you choose to hold mail evaluated by the SMTP message screener, then you might consider moving the location of the Badmail directory. If you server is subject to a spammer’s spam attack, you want to make sure that the held mail doesn’t fill up the operating system partition.

 

Make your configuration changes and click Apply.

 

 

Figure 27

 

5.       Click on the Delivery tab (figure 28). Notice the default entries for the retry intervals. If the SMTP relay is not able to contact the Exchange Server, it will attempt to redeliver the mail based on these intervals. Not that after the third failed attempt, the SMTP relay will continue to try and deliver the mail every 240 minutes. You might want to reduce this value in the event that you need to periodically take the Exchange Server offline for maintenance.

 

Click Apply and then click OK.

 

Figure 28

 

You may have noticed that we didn’t make any configuration changes to the authentication mechanism or the relay characteristics of this virtual SMTP server. The reason is that the default setting is to not relay mail unless a user authenticates. This prevents spammers from hijacking your anonymous inbound SMTP relay while allow users to relay if they authenticate. You’ll create a dedicated virtual SMTP server that will act as both a secure and authenticating SMTP relay.

 

The next step is to configure Remote Domains. You need to create a remote domain for each domain that you want to accept inbound mail for. For example, if you host the mail domains internal.net and domain.com on your Exchange Server, then you need to create a remote domain for internal.net and another remote domain for domain.com. In the current example we’ll create a single remote domain for internal.net.

 

Perform the following steps to create a Remote Domain that you allow anonymous relay to:

 

1.       Open the Internet Information Services (IIS) Manager console and expand your server name (figure 29). Expand the Default SMTP Virtual Server and right click on the Domains node. Point to the New command and click on Domain.

 

Figure 29

 

2.        On the Welcome to the New SMTP Domain Wizard page (figure 30), select the Remote option and click Next.

 

Figure 30

 

3.       On the Domain Name page, type in the name of your mail domain in the Name text box. Click Finish (figure 31).

 

Figure 31

 

4.       Right click on your remote domain in the right pane of the console and click on the Properties command (figure 32).

 

Figure 32

 

5.       On the remote domain’s Properties dialog box, put a checkmark in the Allow incoming mail to be relayed to this domain checkbox. This allows thevirtual SMTP server to relay mail addressed to this remote domain. Remember, this virtual SMTP server does not relay mail and drops all incoming SMTP messages, with the exception being for users who authenticate and for mail addressed to a domain that you’ve configured a remote mail domain for.

 

Select the Forward all mail to smart host option and type in a FQDN or IP address for the Exchange Server on your internal network. If you use a FQDN, make sure this SMTP relay computer can resolve this name to the IP address of the Exchange Server’s virtual SMTP server. If you use an IP address, make sure you surround the address with straight brackets, as seen in figure 33.

 

Click on the Security button.

 

Figure 33

 

6.       By default, this virtual SMTP server does not send credentials to the Exchange Server when it relays mail, and the Exchange Server’s SMTP service does not require credentials. You do have the option of configuring the Exchange Server to require authentication before it will accept the connection from the SMTP relay computer. If you configure the Exchange Server’s SMTP service to require authentication, then you must include valid credentials here. The account and password you enter in this dialog box must match the account you configure on the Exchange Server.

 

In this example, we will allow anonymous connections to the Exchange Server’s SMTP service. The Anonymous access option is select by default and we will leave it at its default. If you make a change on the Outbound Security dialog box (figure 34), click OK. Otherwise, click Cancel.

 

Figure 34

 

 

7.       Close the Internet Information Services console (figure 35).

 

Figure 35

 

 

Create a Second Virtual SMTP Server for Authenticated Inbound Relay for External Users

 

The second virtual SMTP server on the SMTP relay computer is used to allow your external users to relay to both your internal domains and any other domain on the Internet. You do not want to create an anonymous open relay. An open relay can send mail to any domain on the Internet and if it is anonymous, spammers will find your open relay and send gigabytes of spam through it.

 

You can force users to authenticate before they relay. This prevents spammers from using your SMTP relay to send spam, and allows your external users the ability to relay mail to any domain. In addition, you want to secure the messages moving from your external users and the SMTP relay. Some of the information moving from the external user to the SMTP relay may be destined to your internal Exchange Server. These messages may have proprietary information and you want to protect that information from prying eyes.

 

You can prevent intruders from obtaining information from your external users inbound SMTP messages for forcing SSL/TLS encryption. The authenticating SMTP relay can force the external users to negotiate a TLS connection first and after the TLS connection is established, the machine will accept the user’s credentials and accept the transfer of SMTP messages from the external SMTP client. If the external SMTP client does not successfully negotiate a TLS session, then the SMTP relay will drop the connection.

 

Because we want to have two virtual SMTP servers that have different authentication and security requirements, a second IP address will need to be bound to the network interface card.

 

Perform the following steps to add a second IP address to the SMTP relay’s network interface:

 

1.       Right click on the My Network Places icon on the desktop and click on the Properties command (figure 36).

 

Figure 36

 

2.       In the Network Connections windows (figure 37), right click on your network interface and click the Properties command.

 

Figure 37

 

3.       In the connection’s Properties dialog box (figure 38), click on the Internet Protocol (TCP/IP) entry and then click the Properties button.

 

Figure 38

 

4.       In the Internet Protocol (TCP/IP Properties dialog box, click the Advanced button (figure 39).

 

Figure 39

 

5.       In the Advanced TCP/IP Settings dialog box, click the Add button in the IP addresses frame (figure 40).

 

Figure 40

 

6.       Enter the IP address and Subnet mask in the provided text boxes on the TCP/IP Address dialog box (figure 41).

 

Figure 41

 

7.       The second address appears in the IP addresses frame on the IP Settings tab (figure 42). Click OK on the Advanced TCP/IP Settings dialog box and then click OK on the Internet Protocol (TCP/IP) Properties dialog box. Finally, click Close on the connection’s Properties dialog box.

 

Figure 42

 

8.       Close the Network Connections window (figure 43).

 

Figure 43

 

 

Now you can create the second virtual SMTP server and use the second IP address bound to the network interface. Perform the following steps to create the second virtual SMTP server: Its important that each virtual SMTP server listen on a different IP address because you need to have a specific IP address to forward incoming messages from the ISA Server firewall using Server Publishing.

 

Perform the following steps to create the second virtual SMTP server that will be used as an secure authenticating SMTP relay:

 

1.       Open the Internet Information Services (IIS) Manager and expand your server name (figure 44). Right click on the Default SMTP Virtual Server entry, point to New and click on Virtual Server.

 

Figure 44

 

2.       Type in a friendly name for the new virtual SMTP server in the Name text box (figure 45) on the Welcome to the New SMTP Virtual Server Wizard page. You can use nay name you like. This is the name of the virtual SMTP server that will appear in the right pane of the console. Click Next.

 

Figure 45

 

3.       On the Select IP Address page, click the down arrow for the Select the IP address for this SMTP virtual server drop down list box (figure 46). Make sure the IP address is the not the same one used by the anonymous inbound SMTP relay virtual server. Click Next.

 

Figure 46

 

4.       On the Select Home Directory page, type in a path for the home directory of this virtual SMTP server. The Wizard will create the directory if you have not created it already. Enter the path in the Home directory text box (figure 47).

 

Figure 47

 

5.       On the Default Domain page, type in bogus name for a default domain in the Domain text box. You do not want this virtual SMTP server to be an endpoint for any mail so you enter a bogus name. Click Finish.

 

Figure 48

 

6.       Close the Internet Information Services (IIS) Manager console (figure 49).

 

Figure 49

 

 

 

 

Request and Install a Web site Certificate on the Authenticating SMTP Relay Virtual Server (the second virtual SMTP server)

 

The second virtual SMTP server will require the SMTP clients to negotiate a TLS connection before they can authenticate. If the user’s machine cannot establish the TLS sessions, then the connection is dropped before credentials are sent. The TLS session requirement protects user credentials because credentials will never pass between the SMTP client and server without TLS encryption applied.

 

However, before you can configure the virtual SMTP server to force a TLS connection, you must assign the machine a Web certificate. You can get the complete details required to request and assign the virtual SMTP server a Web site certificate in the ISA Server 2000 Exchange Server 2000/2003 Deployment Kit document How to Obtain a Web Site Certificate.

 

 

Configure the Authenticating Virtual SMTP Server for Secure Access

 

We can get to configuring the secure authenticating SMTP relay, now that the second virtual SMTP server has a Web site certificate.

 

Perform the following steps to configure the SMTP virtual server to require authenticating and a secure SSL link:

 

1.       Open the Internet Information Services (IIS) Manager, expand your server name and right click on the second virtual SMTP server. Click on the Properties command (figure 50).

 

Figure 50

 

2.       Click on the Access tab in the second virtual SMTP server’s Properties dialog box (figure 51). Click on the Authentication button in the Access control frame. In this brings up the Authentication dialog box. You have three options:

 

Anonymous access

This option allows any computer to connect to this virtual SMTP server. We do not want this option enabled. We only want authenticated users to connect to this virtual SMTP server

 

Basic authentication

The basic authentication method allows the user name and password to move over the networks in clear text. Anyone with basic network sniffer skills will be able to decode the user name and password sent via basic authentication. However, the basic authentication is a useful authentication method because all SMTP clients support this method of authentication. You can secure user credentials sent via basic authentication by forcing a TLS connection before the credentials are sent.

 

Integrated Windows Authentication

This method allows the virtual SMTP server to accept NTLM authentication. While NTLM authentication is much more secure than basic authentication, there is still the potential for compromise. You can further enhance the improved security of Integrated authentication by forcing TLS on the link.

 

Put a checkmark in the Basic authentication checkbox.

 

Figure 51

 

3.       Read the information in the Basic Authentication dialog box (figure 52). This information reinforces the fact that sending credentials via basic authentication is not secure and that if you choose to use basic authentication, the credentials should be protected by an encrypted link. Click Yes.

 

Figure 52

 

4.       Put a checkmark in the requires TLS encryption checkbox. This causes the virtual SMTP server to require the SMTP client to negotiate a TLS session before basic credentials are accepted. If the SMTP client tries to send credentials via basic authentication without first successfully negotiating a secure link, then the connection attempt will be denied. Enter a default domain for users who send basic credentials in the Default domain text box (figure 53).

 

Put a checkmark in the Integrated Windows Authentication checkbox. This allows the SMTP client to authenticate using NTLM. Note that you do not have the option to require an encrypted link before NTLM authentication is attempted. This is not a problem in our current example, because we will force a secured link for all inbound connections.

 

Click OK in the Authentication dialog box.

 

Figure 53

 

5.       Click on the Communication button in the Secure communication frame (figure 54).

 

Figure 54

 

6.       Put a checkmark in the Require secure channel checkbox in the Security dialog box (figure 55). This forces all incoming connections to this virtual SMTP server to negotiate a TLS connection. Put a checkmark in the Require 128-bit encryption checkbox if you know that all your SMTP clients support 128-bit encryption.

 

Click OK in the Security dialog box.

 

Figure 55

 

 

7.       Click on the Relay button in the Relay restrictions frame (figure 56). In the Relay Restrictions dialog box, confirm that there is a checkmark in the Allow all computers which successfully authenticate to relay, regardless of the list above checkbox. This will allow all your users who successfully authenticate to relay to any domain. You do not need to create any remote domains on this virtual SMTP server because this server will relay to allow domains as long as the virtual SMTP server can resolve the mail domain name to an IP address of an SMTP server responsible for mail for that domain.

 

Click OK ion the Relay Restriction dialog box.

 

Figure 56

 

8.       Click on the Message tab. Configure the sizes limits you prefer. You can remove all size limits by removing the checkmarks from the checkboxes (figure 57). You can also put in an address that receives all non-delivery reports in the Send copy of Non-Delivery Report to. This is helpful if you want to be aware of what addresses are receiving mail that aren’t included in your existing organization. For example, if you host internal.net and mail comes in for a user user1@internal.net. If you don’t have a user1@internal.net, you will receive a non-delivery report for this message. You will not receive a non-delivery report for dropped mail due to a spammer sending mail to your server for domains not under your administrator control.

 

Figure 57

 

9.       Click on the Delivery tab (figure 58). You can customize the retry intervals on this tab. Click the Advanced button.

 

Figure 58

 

10.   On the Advanced Delivery tab, type in the IP address or FQDN in the Exchange Server in the Smart host text box (figure 59). This allows the SMTP relay to forward mail to the Exchange Server for name resolution. You might want to do this to simplify your outbound access scheme.

 

For example, a user logs onto the SMTP server and sends a message destined to somedomain.com. You are not responsible for the somedomain.com email domain. The SMTP server forwards the message to its smart host for email domain resolution. The smart host is the Exchange Server. The Exchange Server then sends a DNS query to its DNS server and resolves the name of the email domain and forwards the messages to the SMTP server responsible for the somedomain.com domain.

 

This simplifies your outbound access scheme because you do not need to create an outbound access rules that allow this SMTP relay outbound access to DNS or SMTP; instead, the Exchange Server had access to these outbound access rules. On the other hand, this increases the amount of traffic and processing cycles required on the Exchange server.

 

In most environments its an acceptable solution to use the Exchange Server as the smart host. However, you should consider performance monitoring on your Exchange Server to confirm that the addition SMTP and DNS traffic does not create an unacceptable load.

 

The Attempt direct delivery option allows you to take advantage of both local mail domain resolution and smart host resolution. In this case, the SMTP relay attempts to resolve the name of the destination mail domain. If the name resolution attempt fails, it forwards the message to its smart host.

 

Click OK.

 

Figure 59

 

11.   Click Apply and then click OK in the Secure Relay Properties dialog box (figure 60).

 

Figure 60

 

12.   Close the Internet Information Services (IIS) Manager console (figure 61).

 

Figure 61

 

Stop and restart the SMTP service. Click on the virtual SMTP server node in the left pane of the console, then click the stop button in the MMC button bar and click the start button on the mmc button bar.

 

 

 

Configure Name Resolution Support for the SMTP Relay

 

The SMTP relay computer needs to be able to resolve Internet mail domains, and optionally, internal network mail domains. If you want the SMTP relay to send mail directly to an internal or external mail domain without using a smart host, then you must configure the machine to use a DNS server that can resolve internal and external mail domain names.

 

The best way to solve this problem is to configure the SMTP relay computer to use a DNS server on your internal network that can resolve both internal and external DNS host names. This allows the SMTP relay to resolve external names so that it can forward mail directly to external SMTP servers when it needs to, and send mail to the Exchange Server when it needs to.

 

Please refer to ISA Server 2000 Exchange Server 2000/2003 Deployment Kit document Configuring DNS to Support Exchange Server Publishing for details on solving DNS issues in an Exchange publishing environment.

 

 

Install the SMTP Message Screener and ISA Management Console on the SMTP Relay

 

We want the SMTP relay computer to screen incoming mail using the ISA Server 2000 SMTP Message Screener. Be aware that the SMTP Message Screener and the SMTP filter are two distinct elements. The SMTP Message Screener looks at the content of the SMTP messages and makes decisions to allow or deny the message based on its content. The SMTP filter protects the internal SMTP server by preventing forwarding of illegal commands and buffer overflow attacks.

 

In order to take advantage of the Message Screener features, you will need to install the SMTP Message Screener on the SMTP relay computer. The SMTP Message Screener will examine all messages moving through it, for all virtual servers.

 

Perform the following steps to install the SMTP Message Screener and ISA Management console on the SMTP relay computer:

 

1.       Double click the isautorun.exe file on the ISA Server 2000 CD-ROM. Click the Install ISA Server icon on the Microsoft ISA Server Setup page (figure 62).

 

Figure 62

 

2.       An ISA 2000 dialog box appears warning you that ISA Server 2000 requires ISA Server 2000 Service Pack 1 to work properly on Windows Server 2003 machines. Click Continue (figure 63).

 

Figure 63

 

3.       Click Continue on the Welcome to the Microsoft ISA Server installation program page (figure 64).

 

Figure 64

 

4.       Enter your CD key into the CD Key text boxes (figure 65). Click OK.

 

Figure 65

 

5.       Write down your product ID and click OK (figure 66).

 

Figure 66

 

6.       Click I Agree on the EULA page after page after you have finished reading the license agreement (figure 67).

 

Figure 67

 

7.       Click the Custom Installation button on the installation type page (figure 68).

 

Figure 68

 

8.       On the options page, remove the checkmark from the ISA Services checkbox (figure 69). Click on the Add-on Services entry and click the Change Option button.

 

Figure 69

 

9.       In the change option dialog box, remove the checkmark from the Install H.323 Gatekeeper Service checkbox and put a checkmark in the Message Screener checkbox (figure 70). Click OK.

 

Figure 70

 

10.   Click Continue in the options dialog box (figure 71).

 

Figure 71

 

11.   In the Launch ISA Management Tool dialog box, remove the checkmark from the Start ISA Server Getting Start Wizard checkbox and click OK (figure 72).

 

Figure 72

 

12.   Click OK on the dialog box informing your that the software was installed successfully (figure 73).

 

Figure 73

 

 

At the point you can install ISA Server SP1 and HF255 and FP1. You must install ISA Server 2000 Service Pack 1 and Hotfix 255. You should also install ISA Server 2000 Feature Pack 1 to insure you have the latest version of the SMTP Message Screener and SMTPcred tool.

 

 

Run the SMTPCred Tool on the SMTP Relay Computer

 

The SMTP Message Screener needs to communicate with the ISA Server firewall to retrieve the SMTP Message Screener settings it uses to allow and deny mail. The SMTP Message Screener must authenticate with the ISA Server firewall before this transaction can take place. The SMTPCred.exe tool allows you to configure the credentials the SMTP Message Screener sends to the ISA Server firewall.

 

Perform the following steps to configure credentials using the SMTPCred tool:

 

1.       Navigate to the Program Files\Microsoft ISA Server folder and double click on the SMTPCRED.EXE icon (figure 74).

 

Figure 74

 

2.       You have the following text boxes you need to fill out on the Message Screener Credentials dialog box (figure 75)

 

ISA Server

Enter the NetBIOS name of the ISA Server firewall in the ISA Server text box. Make sure that the SMTP relay computer can resolve this name to the internal IP address on the ISA Server firewall. You are not able to enter a full FQDN name in this text box, so you will need either a WINS server on your network, or you must configure the SMTP relay with a primary DNS suffix or configure the network interface card on the SMTP relay with a connection specific DNS suffix so that it can append the proper domain name to an unqualified DNS request.

 

Retrieve settings every min

This setting determines how often the SMTP relay will obtain new settings from the ISA Server firewall. For example, if you add a new keyword rule, this setting will determine how long it will take before the SMTP Message Screener on the SMTP relay receives this new information. I suggest setting a low value when you first install and configure the SMTP Message Screener. Once your configuration becomes stable, you should increase the value back to the default of 5 minutes.

 

In the Authentication data frame you have three text boxes:

 

Username

Enter the user name of an administrator on the ISA Server firewall or a domain administrator if the ISA Server firewall belongs to a domain.

 

Domain

If the user name that you entered into the Username text box belongs to a domain, enter the domain name in the Domain text box. If the user is a local administrator on the ISA Server firewall, enter the name of the ISA Server firewall computer.

 

Password

Enter the password for this account in the Password text box.

 

Click OK after you have entered all the information

 

 

Figure 75

 

 

 

Enable the SMTP Filter

 

You do not need to enable the SMTP filter unless you want to allow the ISA Server firewall to protect the published SMTP virtual servers against illegal commands and buffer overflow attacks. The problem with the SMTP filter is that it does not support the STARTTLS command, which is required to create a TLS secured connection between the SMTP client and published SMTP server.

 

If you wish to run a secure authenticating SMTP virtual server and an anonymous inbound relay SMTP server on the internal network, then you will need two ISA Server firewalls if you want to protect the anonymous inbound SMTP relay from illegal commands and buffer overflow attack and allow TLS secured connections to the secure authenticating SMTP virtual server. The reason you can’t use a single ISA Server firewall is that your external users will not be able to create the secure link with the SMTP relay after the SMTP filter is enabled.

 

Note:
You do not need to enable the SMTP filter if you only want to use the SMTP Message Screener. The SMTP Message Screener is separate and distinct from the SMTP filter, although the SMTP Message Screener is configured via the SMTP Filter’s configuration dialog box.

 

Perform the following steps on the SMTP relay computer to enable the SMTP filter on the ISA Server computer that is hosting the SMTP Server Publishing Rule that forwards the SMTP mail to the anonymous inbound SMTP relay virtual server:

 

1.       Open the ISA Management console on the SMTP relay computer (figure 76). Right click on the Internet Security and Acceleration Server node at the top of the left pane and click Connect to. In the Connect to dialog box, type in the NetBIOS name or FQDN for the ISA Server firewall that will be forwarding incoming messages to the anonymous inbound SMTP relay virtual server and click OK.

 

Figure 76

 

2.       In the ISA Management console, expand the Servers and Arrays node and then expand your server name (figure 77). Expand the Extensions node and click on the Application Filters node. Right click on the SMTP Filter node and click Enable.

 

Figure 77

 

3.       Select the Save the change and restart the service(s) option in the ISA Server Warning dialog box that informs you that the Firewall service must be restarted (figure 78). Click OK.

 

Figure 78

 

4.       Notice the icon for the SMTP Filter no longer has the red down arrow.

 

Figure 79

 

 

 

Configure the SMTP Filter and SMTP Message Screener Properties

 

The SMTP filter and SMTP Message Screener are configured using the same interface in the SMTP Filter Properties dialog box. However, keep in mind that the SMTP filter and SMTP Message Screener are two distinct entities. It is possible to use the SMTP filter and not use the SMTP Message Screener and it is possible to use the SMTP Message Screener and not use the SMTP filter.

 

For example, you can use the SMTP Filter without using the SMTP Message Screener by not installing the SMTP Message Screener. The SMTP filter will then protect an published SMTP server against buffer overflow attacks, including the SMTP server co-located on the ISA Server firewall.

 

You can use the SMTP Message Screener and not the SMTP Filter by using a packet filter to allow inbound access to an SMTP relay that is co-located on the ISA Server firewall. The SMTP Message Screener examines the incoming SMTP messages when they are accepted by the IIS 6.0 SMTP service. The SMTP Filter does not protect against buffer overflow attack in this scenario because incoming SMTP messages accepted via a packet filter are not exposed to the SMTP filter.

 

Note:
You must install ISA Server 2000 Feature Pack 1 if you want to support authenticating with a Server Published SMTP server. Pre-Feature Pack 1 versions of the SMTP Filter did not support the AUTH command and would not allow users to authenticate against a Server Published SMTP server. You can authenticate with a Server Published SMTP server after installing Feature Pack 1. Under no circumstances can you use TLS encryption with a Server Published SMTP server when the Message Screener is enabled.

 

Perform the following steps to configure the SMTP filter and SMTP Message Screener components:

 

1.       Open the ISA Management console, expand the Servers and Arrays node and expand your server name. Expand the Extensions node and click on the Application Filters node. Right click on the SMTP Filter entry in the right pane of the console and click on the Properties command (figure 80).

 

Figure 80

 

 

2.       The General tab is the first thing you see when the SMTP Filter Properties dialog box opens (figure 81). You can enable or disable the filter by adding or removing the checkmark in the Enable this filter checkbox. Click on the Keywords tab.

 

Figure 81

 

 

3.       You can enter a prioritized list of keyword to filter on the Keywords tab. The SMTP Message Screener mediates the keyword filtering function. The SMTP filter does not examine SMTP messages for keyword. Click the Add button to add a keyword (figure 82).

 

Figure 82

 

 

4.       Confirm the there is a checkmark in the Enable keyword rule checkbox (figure 83). Type in a keyword that you want the SMTP Message Screener to look for in the Keyword text box. Note that the SMTP Message Screener does not search for whole words; the filter only looks at text strings.

 

Select one of the following options in the Apply action if keyword is found in frame:

 

Message header or body

If the keyword is found in either the message header or message body, then the Action you configure for the rule will be applied.

Message header

If the keyword is found in the header (subject line), then the Action you configure for the rule will be applied.

Message body

If the keyword is found in the body of the message, then the Action you configure for the rule will be applied

 

Click the down arrow for the Action drop down list box. You have the following options:

 

Delete message

The SMTP message is deleted without being saved or informing anyone that it has been deleted.

Hold Message

The SMTP message is held in the BADMAIL directory in the SMTP service’s folder hierarchy. You can view components of the held message, but the message is not saved in a format that you can easily forward to the recipient.

Forward message to

The SMTP message is forwarded to an email address you configure in this rule. Each rule can have a different email address that the message is forwarded to.

 

Click OK on the Mail Keyword Rule dialog box.

 

Figure 83

 

 

5.       The keyword rule appears in the keywords list on the Keywords tab (figure 84). Click on the Users / Domains tab.

 

Figure 84

 

 

6.       You can configure the SMTP Message Screener to block messages based on the sender’s user account or email domain on the Users / Domains tab. Enter a user email account in the Sender’s name text box and click Add. The senders email address appears in the Rejected Sender’s list. Type in a email domain in the Domain name text box and click Add. The email domain appears in the Rejected Domains list.

 

Email messages processed by the SMTP Message Screener matching email addresses or email domains found in these lists are deleted. These messages are not stored anywhere on the server, nor are they forwarded to any user or administrator. If a message from a rejected sender or rejected domain also contains a keyword that matches a keyword rule, and that keyword rule is configured to hold the message, the message will not be held because it is rejected before the keyword search begins.

 

Click Apply and then click OK. Click on the Attachments tab (figure 85).

 

Figure 85

 

 

7.       You can block messages with certain types of attachments on the Attachments tab (figure 86). Click Add to add an attachment rule.

 

Figure 86

 

 

8.       Confirm that there is a checkmark in the Enable attachment rule checkbox on the Mail Attachment Rule dialog box (figure 87). You have three options in the Apply action to messages containing attachments with one of these properties frame:

 

Attachment name

Select this option and type in a name for the attachment, including file name and file extension, in the text box next to this option. Use this option if you don’t want to block all attachments with a particular file extension, but you do want to block a specific file name. For example, you do not want to block all .zip files, but you do want to block a file named exploit.zip.

 

Attachment extension

It is more common to block all files with a specific file extension. For example, if you want to block all files with the exe file extension, select this option and then type in either exe or .exe in the text box to the right of this option.

 

Attachment size limit (in bytes)

You can also block attachments based on their size. Select this option and type in the size of the file extension you want to block.

 

Click the down arrow for the Action drop down list box. You have the following options:

 

Delete message

The SMTP message is deleted without being saved or informing anyone that it has been deleted.

Hold Message

The SMTP message is held in the BADMAIL directory in the SMTP service’s folder hierarchy. You can view components of the held message, but the message is not saved in a format that you can easily forward to the recipient.

Forward message to

The SMTP message is forwarded to an email address you configure in this rule. Each rule can have a different email address that the message is forwarded to.

 

In this example we’ll select the Forward message to option so that you can see how to enter the forwarding address.

 

Figure 87

 

 

9.       When you select the Forward message to option, a text box appears that allows you to enter an email address to forward the message to. However, the server must be able to resolve the address of the mail domain of this user.

 

For example, in figure 88 we have entered the email address smtpsecurityadmin@internal.net. The ISA Server firewall must be able to access an MX record for the internal.net domain. The ISA Server firewall forwards the message to the SMTP server responsible for the internal.net mail based on the information in the MX record.

 

In this example the firewall is configured with a DNS server address of a DNS server on the internal network that can resolve both internal and external network names. The message is forwarded to the internal address of the Exchange server. You must configure a split DNS infrastructure if the internal.net domain is available to both internal and external users.

 

Note:
Please refer to ISA Server 2000 Exchange Server 2000/2003 Deployment Kit document Configuring DNS to Support Exchange Server Publishing for information on how to create a split DNS to support SMTP server publishing.

 

Click OK in the Mail Attachment Rule dialog box. Click on the SMTP Commands tab.

 

Figure 88

 

 

10.   The settings on the SMTP Commands tab are mediated by the SMTP filter component. The SMTP Message Screener does not evaluate SMTP commands and it does not protect against buffer overflow conditions. The commands in the list are limited to a pre-defined length. If an incoming SMTP connection sends a command that exceeds the length allowed, then the connection is dropped. In addition, if a command is sent over the SMTP channel is not on this list, it is dropped.

 

Click the Add button to add an SMTP command to the list (figure 89).

 

Figure 89

 

 

11.   A command you may want to enter into the list of allowed SMTP commands is the AUTH command. This is required if you want to allow external users to authenticate with an SMTP server that is published via an SMTP Server Publishing Rule. Users will not be able to authenticate with a SMTP server Published via an SMTP Server Publishing Rule if the AUTH command is not added to the list and the SMTP filter is enabled.

 

Confirm that the Enable an SMTP command checkbox is checked. Type AUTH in the Command Name text box. Type 1024 in the Maximum Length Bytes text box. Click OK in the SMTP Command Rule dialog box (figure 90).

 

Figure 90

 

 

12.   The new command appears in the list of SMTP commands on the SMTP Commands tab (figure 91). Click Apply and then click OK.

 

Figure 91

 

 

13.   Close the ISA Server Management console.

 

The ISA Server firewall/SMTP server is now ready to filter SMTP messages based on the parameters you set for the SMTP filter and SMTP Message Screener.

 

 

Configure the SMTP Server Publishing Rule on the ISA Server Firewall

 

You can now create your SMTP Server Publishing Rules.

 

1.       Open the ISA Management console, expand the Servers and Arrays node and expand the server name. Expand the Publishing node and click on the Server Publishing Rules node. Right click on the Server Publishing Rules node, point to New and click on Rule (figure 92).

 

Figure 92

 

2.       Type a name for the Sever Publishing Rule in the Server publishing rule name text box on the Welcome to the New Server Publishing Rule Wizard page (figure 93). Click Next.

 

Figure 93

 

3.       Type in the IP address used by the secure authenticating SMTP virtual server in the IP address of internal server text box on the Address Mapping page (figure 94). Click the Browse button under the External IP address on the ISA server and select the IP address you want to use on the external interface of the ISA Server firewall to accept incoming connection requests to the secure authentication SMTP virtual server. Click OK after selecting the address in the New Sever Publishing Rule Wizard dialog box.

 

Figure 94

 

4.       Click Next on the Address Mapping page after the external IP address has been entered (figure 95).

 

Figure 95

 

5.       On the Protocol Settings page (figure 96), click the down arrow from the Apply the rule to this protocol drop down list box and select the SMTP Server Protocol Definition. Click Next.

 

Figure 96

 

6.       On the Client Type page, select the Any request option and click Next (figure 97).

 

Figure 97

 

7.       Review the settings on the Complete the New Server Publishing Rule Wizard page (figure 98), and click Finish.

 

Figure 98

 

 

 

 

Configure the Public DNS to Forward Mail to Your Domains

 

Internet SMTP servers and your remote users must be able to access the published SMTP server using a fully qualified domain name. In addition, if you have users who move between the internal and external network, you should consider using the same domain name for internal and external access. You may not wish to host your publicly available resources using your private domain name, but you should reserve that domain name and configure a public DNS server to resolve Internet accessible resources for your users.

 

The key to making this work is to configure a split DNS infrastructure. The split DNS allows your roving users to access mail resources while on the road, or on the internal network, and never need to change mail client or browser settings. Please refer to ISA Server 2000 Exchange Server 2000/2003 Deployment Kit document Configuring DNS to Support Exchange Server Publishing