Configuring a Secure Internal SMTP
Relay with the SMTP Filter and Message Screener
An SMTP
relay is a machine that accepts SMTP messages for mail domains for which is it
not authoritative. The most common example of an SMTP relay is an SMTP server
run by an ISP that accepts SMTP messages destined to other domains. The user
logs onto his ISP and his email client software is configured
to use the SMTP server at his ISP. The user sends an SMTP message to a user at domain.com, which is not run by the ISP
the user connects to. The SMTP message is sent to the ISP’s SMTP server and the ISP’s SMTP server relays the mail to the appropriate SMTP
server for the domain.com domain.
You can
create an SMTP relay on your internal network that accepts incoming mail to the
domains that you are responsible for. This relay can leverage the ISA Server
2000 SMTP Message Screener to block messages based on source address or domain,
attachment characteristics or keywords. In addition, you can configure a
second, secure authenticating SMTP virtual server on the SMTP relay computer to
allow your external users to relay mail to any domain.
The primary
advantage of using an SMTP relay for incoming connections is that you avoid
allowing anonymous external computers from creating new inbound SMTP
connections to your Exchange Server. The incoming connections are made to the SMTP relay and the SMTP relay forwards
legitimate mail to email domains hosted on the Exchange Server to the Exchange
Server machine. In addition, the SMTP relay machine absorbs the processing
costs involved with the deep inspection of SMTP messages that the ISA Server
2000 SMTP filter performs.
You need to
carry out the following procedures to create an anonymous inbound SMTP relay
that sends mail to domains under your administrative control and create a
secure authenticating SMTP relay your external users can use to relay mail to
all mail domains:
The
remainder of the ISA Server 2000
Exchange Server 2000/2003 Deployment Kit document covers each of these
procedures in detail.
Installing Windows Server 2003 on
the Firewall Computer
The computer
that becomes the ISA Server 2000 firewall must meet the following minimum
requirements:
The ISA
Server firewall and Web caching components work very well on very modest
hardware. This is true even when the SMTP filter is enabled
and protecting the published co-located SMTP server. However, the SMTP Message
Screener can be very processor intensive. This is why I recommend that you use
a processor with a minimum of rating of 1.5 MHz. This is especially true if you
plan on running an authenticating and non-authenticating SMTP relay on the same
computer.
Installing ISA Server 2000 on the
Firewall Computer
Requirements
and procedures for installing ISA Server 2000 on a Windows Server 2003 are
slightly different from those for installing ISA Server 2000 on a Windows 2000
computer. Please refer to ISA Server
2000 Exchange Server 2000/2003 Deployment Kit document Installing ISA Server 2000 on
Windows Server 2003 for details on how to install ISA Server
2000 on a Windows Server 2003 computer.
Configure DCOM Permissions on the
Firewall Computer
The SMTP relay on the internal network using DCOM to communicate with the
SMTP filter components on the ISA Sever firewall. DCOM permissions must be configured to allow the SMTP relay on the internal
network to communicate with the filter. Perform the following steps to
configure the proper DCOM permissions:
Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Install Windows Server 2003 on the
SMTP Relay Computer
The SMTP
relay computer must meet the basic hardware requirements for installing Windows
Server 2003. In addition, consider the following options for the SMTP relay
machine:
In this ISA Server 2000 Exchange Server 2000/2003
Deployment Kit article, we will assume that the machine is not a member of the user domain and that
you will assign the users a group account to authenticate with the secure
authenticating SMTP server. If the SMTP relay were a member of the Windows
domain that contained user accounts, you might consider allowing users to use
their own user account to authenticate with the SMTP relay computer.
Install the SMTP Service on the SMTP
Relay Computer
For
security reasons, IIS 6.0 is not installed by default
on a Windows Server 2003 computer. You must install the IIS 6.0 SMTP service on
the Windows Server 2003 SMTP relay.
Perform the
following steps to install the IIS 6.0 SMTP relay computer:
1.
Click Start, point to Control
Panel and click on Add or Remove
Programs. In the Add or Remove
Programs window, click the Add/Remove
Windows Components button. You will see a Windows Setup dialog box asking your to
please wait (figure 16).
Figure 16

2.
In the Windows Components dialog box (figure 17), click on the Application Server entry (do not put a
checkmark in its checkbox!). Click Details.
Figure 17

3.
In the Application Server dialog box (figure 18), Click on the Internet Information Services (IIS)
entry (do not put a checkmark in its checkbox!). Click Details.
Figure 18
(fig18)

4.
In the Internet Information Services (IIS) dialog box, put a checkmark in
the SMTP Service checkbox (figure
19). Click OK.
Figure 19

5.
Click OK in the Application Server
dialog box (figure 20).
Figure 20

6.
Click Next on the Windows Components dialog box (figure 21).
Figure 21

7.
A progress bar appears as the
application installs (figure 22).
Figure 22

8.
Click Finish on the Completing the
Windows Components Wizard page (figure 23).
Figure 23

Configure the First Virtual SMTP
Server (Non-Authenticating) for Anonymous Inbound Relay
We will run
both an anonymous inbound SMTP relay that allows Internet SMTP servers to relay
SMTP mail to the domains under your administrative control and a secure authenticating SMTP relay that you external users can
use to relay mail to all domains, internal and external. We will configure the Default SMTP Virtual Service to provide
the anonymous inbound relay. Later we will install a second virtual SMTP server
that you will use as the secure authenticating SMTP relay.
1.
Click Start and point to Administrative
Tools. Click on the Internet
Information Services (IIS) Manager entry (figure 24).
Figure 24

2.
Right click on the Default SMTP Virtual Server entry in
the left pane of the console (figure 25). Click the Properties command.
Figure 25

3.
In the Default SMTP Virtual Server Properties dialog box (figure 26),
click the General tab. Click the
down arrow for the IP address drop
down list box and select the IP address that you want the anonymous inbound
SMTP relay to use. This anonymous relay accepts mail for the mail domains that
are under your administrative control and drops all other inbound mail.
Internet SMTP servers use this machine to send mail to the Exchange Server.
Click Apply after selecting the IP
address.
Figure 26

4.
Click on the Messages tab (figure 27). You have the following options:
Limit message size to
(KB)
This is what Microsoft SMTP Service will advertise, in
kilobytes, as the maximum message size this SMTP virtual server will accept. If
a mail client sends a message that exceeds the limit, the client will get an
error. If a remote server supports EHLO, it will
detect the advertised maximum message size value when it connects to the SMTP
virtual server and won't even attempt to deliver a message that exceeds the
limit. Instead it will simply NDR the sender of the
message. A remote server that doesn't support EHLO
will try to send a message that exceeds the size limit, but will still end up
sending an NDR to the sender when the message doesn't go through. The default
is 2048 KB. The minimum value is 1KB. To have no limit, clear the check box.
Limit session size to
(KB)
This is the maximum amount of data, in kilobytes, accepted
during the total connection. It is the sum of all messages sent during the
connection (applying to the message body only). Type a value larger than the
Limit message size to (KB). This maximum should be set carefully, because the
connecting message transfer agent (MTA) is likely to
resubmit the message repeatedly. The default size is 10240 KB. This value
should be greater than or equal to the value entered for Limit message size to
(KB). To have no limit, clear the check box.
Limit number of
message per connection to
When the check box is selected,
this option enables you to limit the number of messages sent in a single
connection. The default is 20. This feature also provides a method to increase
system performance by using multiple connections to deliver messages to a
remote domain. Consequently, once the set limit is reached,
a new connection is automatically opened and the transmission continues until
all messages are delivered. To disable this feature and have no limit, clear
the check box.
Limit number of
recipients per message to
This setting limits the maximum number of recipients for a
single message. The default is 100, which is the minimum required number
specified in Request for Comments (RFC) 821. To disable this feature and have
no limit, clear the check box. Some clients return messages with a non-delivery
report (NDR) once an error message is received
indicating that the maximum number of recipients has been exceeded. A server
running Microsoft SMTP Service does not return messages with an NDR in this
instance. It opens a new connection immediately and processes the remaining
recipients. For example, if the recipient limit is set to 100 and a message
with 105 recipients is being transmitted, the first
100 are delivered in one connection after receipt of the error message. Then a
new connection is opened and the message is processed
for the remaining five recipients.
Send copy of
Non-Delivery Report to
When a message is undeliverable, it is
returned to the sender with a non-delivery report (NDR). You can
designate that copies of the NDR are sent to a
specific SMTP mailbox. Type an e-mail address for the mailbox.
Badmail Directory
When a message is undeliverable, it is
returned to the sender with a non-delivery report (NDR). You can
designate that copies of the NDR are sent to a
location of your choice.
All NDRs go through the same delivery process as other
messages, including attempts to resend the message. If the NDR has reached the
retry limit and cannot be delivered to the sender, a
copy of the message is placed in the Badmail directory. Messages placed in the
Badmail directory cannot be delivered or returned.
Check the directory regularly and reconcile the messages, because a full directory
may adversely affect Microsoft SMTP Service performance.
If you choose to hold mail evaluated by the SMTP message
screener, then you might consider moving the location of the Badmail directory.
If you server is subject to a spammer’s spam attack, you want to make sure that
the held mail doesn’t fill up the operating system partition.
Make your configuration changes and click Apply.
Figure 27

5.
Click on the Delivery tab (figure 28). Notice the default entries for the retry
intervals. If the SMTP relay is not able to contact the Exchange Server, it
will attempt to redeliver the mail based on these intervals. Not that after the
third failed attempt, the SMTP relay will continue to try and deliver the mail
every 240 minutes. You might want to reduce this value in the event that you
need to periodically take the Exchange Server offline for maintenance.
Click Apply and
then click OK.
Figure 28

You may
have noticed that we didn’t make any configuration changes to the
authentication mechanism or the relay characteristics of this virtual SMTP
server. The reason is that the default setting is to not relay mail unless a user authenticates. This prevents spammers
from hijacking your anonymous inbound SMTP relay while allow users to relay if
they authenticate. You’ll create a dedicated virtual SMTP server that will act
as both a secure and authenticating SMTP relay.
The next
step is to configure Remote Domains.
You need to create a remote domain for each domain that you want to accept
inbound mail for. For example, if you host the mail domains internal.net and domain.com on your Exchange Server, then you need to create a
remote domain for internal.net and
another remote domain for domain.com.
In the current example we’ll create a single remote domain for internal.net.
Perform the
following steps to create a Remote Domain that you allow anonymous relay to:
1.
Open the Internet Information Services (IIS) Manager console and expand your
server name (figure 29). Expand the Default
SMTP Virtual Server and right click on the Domains node. Point to the New command and click on Domain.
Figure 29

2.
On the Welcome
to the New SMTP Domain Wizard page (figure 30), select the Remote option and click Next.
Figure 30

3.
On the Domain Name page, type in the name of your mail domain in the Name text box. Click Finish (figure 31).
Figure 31

4.
Right click on your remote domain in
the right pane of the console and click on the Properties command (figure 32).
Figure 32

5.
On the remote domain’s Properties dialog box, put a checkmark
in the Allow incoming mail to be relayed to this domain checkbox. This allows thevirtual SMTP server to relay mail addressed to this
remote domain. Remember, this virtual SMTP server does not relay mail and drops all
incoming SMTP messages, with the exception being for users who authenticate and
for mail addressed to a domain that you’ve configured a remote mail domain for.
Select the Forward
all mail to smart host option and type in a FQDN or IP address for the
Exchange Server on your internal network. If you use a FQDN, make sure this
SMTP relay computer can resolve this name to the IP address of the Exchange
Server’s virtual SMTP server. If you use an IP address, make sure you surround
the address with straight brackets, as seen in figure 33.
Click on the Security
button.
Figure 33

6.
By default, this virtual SMTP server
does not send credentials to the Exchange Server when it relays mail, and the
Exchange Server’s SMTP service does not require credentials. You do have the
option of configuring the Exchange Server to require authentication before it
will accept the connection from the SMTP relay computer. If you configure the
Exchange Server’s SMTP service to require authentication, then you must include
valid credentials here. The account and password you enter in this dialog box
must match the account you configure on the Exchange Server.
In this example, we will allow anonymous connections to the
Exchange Server’s SMTP service. The Anonymous
access option is select by default and we will leave it at its default. If
you make a change on the Outbound
Security dialog box (figure 34), click OK.
Otherwise, click Cancel.
Figure 34

7.
Close the Internet Information Services console (figure 35).
Figure 35

Create a Second Virtual SMTP Server
for Authenticated Inbound Relay for External Users
The second
virtual SMTP server on the SMTP relay computer is used
to allow your external users to relay to both your internal domains and any
other domain on the Internet. You do not want to create an anonymous open
relay. An open relay can send mail to any domain on the Internet and if it is
anonymous, spammers will find your open relay and send gigabytes of spam
through it.
You can
force users to authenticate before they relay. This prevents spammers from
using your SMTP relay to send spam, and allows your external users the ability
to relay mail to any domain. In addition, you want to secure the messages
moving from your external users and the SMTP relay. Some of the information
moving from the external user to the SMTP relay may be destined to your
internal Exchange Server. These messages may have proprietary information and
you want to protect that information from prying eyes.
You can
prevent intruders from obtaining information from your external users inbound SMTP messages for forcing SSL/TLS encryption.
The authenticating SMTP relay can force the external users to negotiate a TLS
connection first and after the TLS connection is established, the machine will
accept the user’s credentials and accept the transfer of SMTP messages from the
external SMTP client. If the external SMTP client does not successfully
negotiate a TLS session, then the SMTP relay will drop the connection.
Because we
want to have two virtual SMTP servers that have different authentication and
security requirements, a second IP address will need to be bound to the network
interface card.
Perform the
following steps to add a second IP address to the SMTP relay’s network
interface:
1.
Right click on the My Network Places icon on the desktop
and click on the Properties command
(figure 36).
Figure 36

2.
In the Network Connections windows (figure 37), right click on your
network interface and click the Properties
command.
Figure 37

3.
In the connection’s Properties dialog box (figure 38),
click on the Internet Protocol (TCP/IP)
entry and then click the Properties
button.
Figure 38

4.
In the Internet Protocol (TCP/IP Properties dialog box, click the Advanced button (figure 39).
Figure 39

5.
In the Advanced TCP/IP Settings dialog box, click the Add button in the IP
addresses frame (figure 40).
Figure 40

6.
Enter the IP address and Subnet mask
in the provided text boxes on the TCP/IP
Address dialog box (figure 41).
Figure 41

7.
The second address appears in the IP addresses frame on the IP Settings tab (figure 42). Click OK on the Advanced TCP/IP Settings dialog box and then click OK on the Internet Protocol (TCP/IP) Properties dialog box. Finally, click Close on the connection’s Properties dialog box.
Figure 42

8.
Close the Network Connections window (figure 43).
Figure 43

Now you can
create the second virtual SMTP server and use the second IP address bound to
the network interface. Perform the following steps to create the second virtual
SMTP server: Its important that each virtual SMTP
server listen on a different IP address because you need to have a specific IP
address to forward incoming messages from the ISA Server firewall using Server
Publishing.
Perform the
following steps to create the second virtual SMTP server that will be used as an secure authenticating SMTP relay:
1.
Open the Internet Information Services (IIS) Manager and expand your server
name (figure 44). Right click on the Default
SMTP Virtual Server entry, point to New
and click on Virtual Server.
Figure 44

2.
Type in a friendly name for the new
virtual SMTP server in the Name text
box (figure 45) on the Welcome to the
New SMTP Virtual Server Wizard page. You can use nay name you like. This is
the name of the virtual SMTP server that will appear in the right pane of the
console. Click Next.
Figure 45

3.
On the Select IP Address page, click the down arrow for the Select the IP address for this SMTP virtual
server drop down list box (figure 46). Make sure the IP address is the not the same one used by the anonymous
inbound SMTP relay virtual server. Click Next.
Figure 46

4.
On the Select Home Directory page, type in a path for the home directory
of this virtual SMTP server. The Wizard will create the directory if you have
not created it already. Enter the path in the Home directory text box (figure 47).
Figure 47

5.
On the Default Domain page, type in bogus name for a default domain in the
Domain text box. You do not want
this virtual SMTP server to be an endpoint for any mail so you enter a bogus
name. Click Finish.
Figure 48

6.
Close the Internet Information Services (IIS)
Manager console (figure 49).
Figure 49

Request and Install a Web site
Certificate on the Authenticating SMTP Relay Virtual Server (the second virtual
SMTP server)
The second
virtual SMTP server will require the SMTP clients to negotiate a TLS connection
before they can authenticate. If the user’s machine cannot establish the TLS
sessions, then the connection is dropped before
credentials are sent. The TLS session requirement protects user credentials
because credentials will never pass between the SMTP client and server without
TLS encryption applied.
However,
before you can configure the virtual SMTP server to force a TLS connection, you
must assign the machine a Web certificate. You can get the complete details
required to request and assign the virtual SMTP server a Web site certificate
in the ISA Server 2000 Exchange Server
2000/2003 Deployment Kit document
How to Obtain a Web Site Certificate.
Configure the Authenticating Virtual
SMTP Server for Secure Access
We can get
to configuring the secure authenticating SMTP relay,
now that the second virtual SMTP server has a Web site certificate.
Perform the
following steps to configure the SMTP virtual server to require authenticating
and a secure SSL link:
1.
Open the Internet Information Services (IIS) Manager, expand your server
name and right click on the second virtual SMTP server. Click on the Properties command (figure 50).
Figure 50

2.
Click on the Access tab in the second virtual SMTP server’s Properties dialog box (figure 51). Click on the Authentication button in the Access control frame. In this brings up
the Authentication dialog box. You
have three options:
Anonymous access
This option allows any computer to connect to this virtual
SMTP server. We do not want this
option enabled. We only want authenticated users to connect to this virtual
SMTP server
Basic authentication
The basic authentication method allows the user name and
password to move over the networks in clear text. Anyone with basic network
sniffer skills will be able to decode the user name and password sent via basic
authentication. However, the basic authentication is a useful authentication
method because all SMTP clients support this method of authentication. You can
secure user credentials sent via basic authentication by forcing a TLS
connection before the credentials are sent.
Integrated Windows
Authentication
This method allows the virtual SMTP server to accept NTLM
authentication. While NTLM authentication is much more secure than basic
authentication, there is still the potential for compromise. You can further
enhance the improved security of Integrated
authentication by forcing TLS on the link.
Put a checkmark in the Basic
authentication checkbox.
Figure 51

3.
Read the information in the Basic Authentication dialog box (figure
52). This information reinforces the fact that sending credentials via basic
authentication is not secure and that if you choose to use basic
authentication, the credentials should be protected by
an encrypted link. Click Yes.
Figure 52

4.
Put a checkmark in the requires TLS encryption checkbox. This causes
the virtual SMTP server to require the SMTP client to negotiate a TLS session before basic credentials are accepted.
If the SMTP client tries to send credentials via basic authentication without
first successfully negotiating a secure link, then the connection attempt will be denied. Enter a default domain for users who send
basic credentials in the Default domain
text box (figure 53).
Put a checkmark in the Integrated
Windows Authentication checkbox. This allows the SMTP client to
authenticate using NTLM. Note that you do not have the option to require an
encrypted link before NTLM authentication is attempted. This is not a problem
in our current example, because we will force a secured link for all inbound
connections.
Click OK in the Authentication dialog box.
Figure 53

5.
Click on the Communication button in the Secure communication
frame (figure 54).
Figure 54

6.
Put a checkmark in the Require secure channel checkbox in the Security dialog box (figure 55). This
forces all incoming connections to this virtual SMTP server to negotiate a TLS
connection. Put a checkmark in the Require 128-bit
encryption checkbox if you know that all your SMTP clients support 128-bit
encryption.
Click OK in the Security dialog box.
Figure 55

7.
Click on the Relay button in the Relay
restrictions frame (figure 56). In the Relay
Restrictions dialog box, confirm that there is a checkmark in the Allow all computers which successfully
authenticate to relay, regardless of the list above checkbox. This will
allow all your users who successfully authenticate to relay to any domain. You
do not need to create any remote domains on this virtual SMTP server because
this server will relay to allow domains as long as the virtual SMTP server can resolve
the mail domain name to an IP address of an SMTP server responsible for mail
for that domain.
Click OK ion the Relay Restriction dialog box.
Figure 56

8.
Click on the Message tab. Configure the sizes limits you prefer. You can remove
all size limits by removing the checkmarks from the checkboxes (figure 57). You
can also put in an address that receives all non-delivery reports in the Send copy of Non-Delivery Report to.
This is helpful if you want to be aware of what addresses are receiving mail that
aren’t included in your existing organization. For
example, if you host internal.net and
mail comes in for a user user1@internal.net. If you don’t have
a user1@internal.net, you will receive
a non-delivery report for this message. You will not receive a non-delivery
report for dropped mail due to a spammer sending mail to your server for
domains not under your administrator control.
Figure 57

9.
Click on the Delivery tab (figure 58). You can customize the retry intervals on
this tab. Click the Advanced button.
Figure 58

10. On the Advanced Delivery tab, type in the IP address or FQDN in the
Exchange Server in the Smart host
text box (figure 59). This allows the SMTP relay to forward mail to the
Exchange Server for name resolution. You might want to do this to simplify your
outbound access scheme.
For example, a user logs onto the SMTP server and sends a
message destined to somedomain.com.
You are not responsible for the somedomain.com
email domain. The SMTP server forwards the message to its smart host for email
domain resolution. The smart host is the Exchange Server. The Exchange Server
then sends a DNS query to its DNS server and resolves the name of the email
domain and forwards the messages to the SMTP server responsible for the somedomain.com domain.
This simplifies your outbound access scheme because you do
not need to create an outbound access rules that allow this SMTP relay outbound
access to DNS or SMTP; instead, the Exchange Server had access to these
outbound access rules. On the other hand, this increases the amount of traffic
and processing cycles required on the Exchange server.
In most environments its an
acceptable solution to use the Exchange Server as the smart host. However, you
should consider performance monitoring on your Exchange Server to confirm that
the addition SMTP and DNS traffic does not create an unacceptable load.
The Attempt direct
delivery option allows you to take advantage of both local mail domain
resolution and smart host resolution. In this case, the SMTP relay attempts to
resolve the name of the destination mail domain. If the name resolution attempt
fails, it forwards the message to its smart host.
Click OK.
Figure 59

11. Click Apply and then click OK
in the Secure Relay Properties
dialog box (figure 60).
Figure 60

12. Close the Internet Information Services (IIS) Manager console (figure 61).
Figure 61

Stop and
restart the SMTP service. Click on the virtual SMTP server node in the left
pane of the console, then click the stop button in the MMC button bar and click
the start button on the mmc button bar.
Configure Name Resolution Support
for the SMTP Relay
The SMTP
relay computer needs to be able to resolve Internet mail domains, and
optionally, internal network mail domains. If you want the SMTP relay to send
mail directly to an internal or external mail domain without using a smart
host, then you must configure the machine to use a DNS server that can resolve
internal and external mail domain names.
The best
way to solve this problem is to configure the SMTP relay computer to use a DNS
server on your internal network that can resolve both internal and external DNS
host names. This allows the SMTP relay to resolve external names so that it can
forward mail directly to external SMTP servers when it needs to, and send mail
to the Exchange Server when it needs to.
Please
refer to ISA Server 2000 Exchange Server
2000/2003 Deployment Kit document
Configuring DNS to Support Exchange Server Publishing
for details on solving DNS issues in an Exchange publishing environment.
Install the SMTP Message Screener
and ISA Management Console on the SMTP Relay
We want the
SMTP relay computer to screen incoming mail using the ISA Server 2000 SMTP
Message Screener. Be aware that the SMTP Message Screener and the SMTP filter
are two distinct elements. The SMTP Message Screener looks at the content of
the SMTP messages and makes decisions to allow or deny the message based on its
content. The SMTP filter protects the internal SMTP server by preventing
forwarding of illegal commands and buffer overflow attacks.
In order to
take advantage of the Message Screener features, you will need to install the
SMTP Message Screener on the SMTP relay computer. The SMTP Message Screener
will examine all messages moving through it, for all virtual servers.
Perform the
following steps to install the SMTP Message Screener and ISA Management console
on the SMTP relay computer:
1.
Double click the isautorun.exe file on the ISA Server
2000 CD-ROM. Click the Install ISA
Server icon on the Microsoft ISA
Server Setup page (figure 62).
Figure 62

2.
An ISA 2000 dialog box appears warning you that ISA Server 2000
requires ISA Server 2000 Service Pack 1 to work properly on Windows Server 2003
machines. Click Continue (figure
63).
Figure 63

3.
Click Continue on the Welcome to
the Microsoft ISA Server installation program page (figure 64).
Figure 64

4.
Enter your CD key into the CD Key text boxes (figure 65). Click OK.
Figure 65

5.
Write down your product ID and click
OK (figure 66).
Figure 66

6.
Click I Agree on the EULA page after page after
you have finished reading the license agreement (figure 67).
Figure 67

7.
Click the Custom Installation button on the installation type page (figure
68).
Figure 68

8.
On the options page, remove the
checkmark from the ISA Services
checkbox (figure 69). Click on the Add-on
Services entry and click the Change
Option button.
Figure 69

9.
In the change option dialog box,
remove the checkmark from the Install
H.323 Gatekeeper Service checkbox and put a checkmark in the Message Screener checkbox (figure 70).
Click OK.
Figure 70

10. Click Continue in the options dialog box (figure 71).
Figure 71

11. In the Launch ISA Management Tool dialog box, remove the checkmark from
the Start ISA Server Getting Start
Wizard checkbox and click OK
(figure 72).
Figure 72

12. Click OK on the dialog box informing your that
the software was installed successfully (figure 73).
Figure 73

At the
point you can install ISA Server SP1 and HF255 and FP1. You must install ISA
Server 2000 Service Pack 1 and Hotfix 255. You should also install ISA Server
2000 Feature Pack 1 to insure you have the latest version of the SMTP Message
Screener and SMTPcred tool.
Run the SMTPCred Tool on the SMTP Relay Computer
The SMTP
Message Screener needs to communicate with the ISA Server firewall to retrieve
the SMTP Message Screener settings it uses to allow and deny mail. The SMTP
Message Screener must authenticate with the ISA Server firewall before this
transaction can take place. The SMTPCred.exe
tool allows you to configure the credentials the SMTP Message Screener sends to
the ISA Server firewall.
Perform the
following steps to configure credentials using the SMTPCred tool:
1.
Navigate to the Program Files\Microsoft ISA Server folder and double click on the SMTPCRED.EXE icon (figure 74).
Figure 74

2.
You have the following text boxes
you need to fill out on the Message
Screener Credentials dialog box (figure 75)
ISA Server
Enter the NetBIOS name of the ISA Server firewall in the ISA Server text box. Make sure that the
SMTP relay computer can resolve this name to the internal IP address on the ISA
Server firewall. You are not able to enter a full FQDN name in this text box,
so you will need either a WINS server on your network, or you must configure
the SMTP relay with a primary DNS suffix or configure the network interface
card on the SMTP relay with a connection specific DNS suffix so that it can
append the proper domain name to an unqualified DNS request.
Retrieve settings
every min
This setting determines how often the SMTP relay will obtain
new settings from the ISA Server firewall. For example, if you add a new
keyword rule, this setting will determine how long it will take before the SMTP
Message Screener on the SMTP relay receives this new information. I suggest
setting a low value when you first install and configure the SMTP Message
Screener. Once your configuration becomes stable, you should increase the value
back to the default of 5 minutes.
In the Authentication
data frame you have three text boxes:
Username
Enter the user name of an administrator on the ISA Server
firewall or a domain administrator if the ISA Server firewall belongs to a
domain.
Domain
If the user name that you entered into the Username text box belongs to a domain,
enter the domain name in the Domain
text box. If the user is a local administrator on the ISA Server firewall,
enter the name of the ISA Server firewall computer.
Password
Enter the password for this account in the Password text box.
Click OK after
you have entered all the information
Figure 75

Enable the SMTP Filter
You do not
need to enable the SMTP filter unless you want to allow the ISA Server firewall
to protect the published SMTP virtual servers against illegal commands and
buffer overflow attacks. The problem with the SMTP filter is that it does not
support the STARTTLS command, which is required to create a TLS secured
connection between the SMTP client and published SMTP server.
If you wish
to run a secure authenticating SMTP virtual server and an anonymous inbound
relay SMTP server on the internal network, then you will need two ISA Server firewalls if you want to
protect the anonymous inbound SMTP relay from illegal commands and buffer
overflow attack and allow TLS secured connections to the secure authenticating
SMTP virtual server. The reason you can’t use a single ISA Server firewall is
that your external users will not be able to create the secure link with the
SMTP relay after the SMTP filter is enabled.
Note:
You do not need to enable the SMTP filter if you only want to use the SMTP
Message Screener. The SMTP Message Screener is separate and distinct from the
SMTP filter, although the SMTP Message Screener is configured
via the SMTP Filter’s configuration dialog box.
Perform the
following steps on the SMTP relay computer to enable the SMTP filter on the ISA
Server computer that is hosting the SMTP Server Publishing Rule that forwards
the SMTP mail to the anonymous inbound SMTP relay virtual server:
1.
Open the ISA Management console on the SMTP relay computer (figure 76).
Right click on the Internet Security and
Acceleration Server node at the top of the left pane and click Connect to. In the Connect to dialog box, type in the NetBIOS name or FQDN for the ISA
Server firewall that will be forwarding incoming messages to the anonymous
inbound SMTP relay virtual server and click OK.
Figure 76

2.
In the ISA Management console, expand the Servers and Arrays node and then expand your server name (figure
77). Expand the Extensions node and
click on the Application Filters
node. Right click on the SMTP Filter
node and click Enable.
Figure 77

3.
Select the Save the change and restart the service(s) option in the ISA Server Warning dialog box that
informs you that the Firewall service must be restarted (figure 78). Click OK.
Figure 78

4.
Notice the icon for the SMTP Filter no longer has the red down
arrow.
Figure 79

Configure the SMTP Filter and SMTP
Message Screener Properties
The SMTP
filter and SMTP Message Screener are configured using
the same interface in the SMTP Filter
Properties dialog box. However, keep
in mind that the SMTP filter and SMTP Message Screener are two distinct
entities. It is possible to use the SMTP filter and not use the SMTP Message
Screener and it is possible to use the SMTP Message Screener and not use the
SMTP filter.
For
example, you can use the SMTP Filter without using the SMTP Message Screener by
not installing the SMTP Message Screener. The SMTP filter will then protect an published SMTP server against buffer overflow attacks,
including the SMTP server co-located on the ISA Server firewall.
You can use
the SMTP Message Screener and not the SMTP Filter by using a packet filter to
allow inbound access to an SMTP relay that is co-located on the ISA Server
firewall. The SMTP Message Screener examines the incoming SMTP messages when
they are accepted by the IIS 6.0 SMTP service. The
SMTP Filter does not protect against buffer overflow attack in this scenario
because incoming SMTP messages accepted via a packet filter are
not exposed to the SMTP filter.
Note:
You must install ISA Server 2000 Feature Pack 1 if you want to support
authenticating with a Server Published SMTP server. Pre-Feature Pack 1 versions
of the SMTP Filter did not support the AUTH command and would not allow users
to authenticate against a Server Published SMTP server. You can authenticate
with a Server Published SMTP server after installing Feature Pack 1. Under no
circumstances can you use TLS encryption with a Server Published SMTP server
when the Message Screener is enabled.
Perform the
following steps to configure the SMTP filter and SMTP Message Screener
components:
1.
Open the ISA Management console, expand the Servers and Arrays node and expand your server name. Expand the Extensions node and click on the Application Filters node. Right click
on the SMTP Filter entry in the
right pane of the console and click on the Properties
command (figure 80).
Figure 80

2.
The General tab is the first thing you see when the SMTP Filter Properties dialog box opens
(figure 81). You can enable or disable the filter by adding or removing the
checkmark in the Enable this filter
checkbox. Click on the Keywords tab.
Figure 81

3.
You can enter a prioritized list of
keyword to filter on the Keywords
tab. The SMTP Message Screener mediates the keyword filtering function. The
SMTP filter does not examine SMTP
messages for keyword. Click the Add
button to add a keyword (figure 82).
Figure 82

4.
Confirm the there is a checkmark in
the Enable keyword rule checkbox
(figure 83). Type in a keyword that you want the SMTP Message Screener to look
for in the Keyword text box. Note
that the SMTP Message Screener does not search for whole words; the filter only
looks at text strings.
Select one of the following options in the Apply action if keyword is
found in frame:
Message header or body
If the keyword is found in either the message header or message
body, then the Action you configure
for the rule will be applied.
Message header
If the keyword is found in the
header (subject line), then the Action
you configure for the rule will be applied.
Message body
If the keyword is found in the body
of the message, then the Action you
configure for the rule will be applied
Click the down arrow for the Action drop down list box. You have the following options:
Delete message
The SMTP message is deleted without
being saved or informing anyone that it has been deleted.
Hold Message
The SMTP message is held in the BADMAIL directory in the SMTP service’s
folder hierarchy. You can view components of the held message, but the message is not saved in a format that you can easily forward to the
recipient.
Forward message to
The SMTP message is forwarded to an
email address you configure in this rule. Each rule can have a different email
address that the message is forwarded to.
Click OK on the Mail Keyword Rule dialog box.
Figure 83

5.
The keyword rule appears in the
keywords list on the Keywords tab
(figure 84). Click on the Users /
Domains tab.
Figure 84

6.
You can configure the SMTP Message
Screener to block messages based on the sender’s user account or email domain
on the Users / Domains tab. Enter a
user email account in the Sender’s name
text box and click Add. The senders
email address appears in the Rejected
Sender’s list. Type in a email domain in the Domain name text box and click Add. The email domain appears in the Rejected Domains list.
Email messages processed by the SMTP Message Screener
matching email addresses or email domains found in these lists are deleted. These messages are not stored anywhere on the
server, nor are they forwarded to any user or administrator. If a message from
a rejected sender or rejected domain also contains a keyword that matches a
keyword rule, and that keyword rule is configured to
hold the message, the message will not be held because it is rejected before
the keyword search begins.
Click Apply and
then click OK. Click on the Attachments tab (figure 85).
Figure 85

7.
You can block messages with certain
types of attachments on the Attachments
tab (figure 86). Click Add to add an
attachment rule.
Figure 86

8.
Confirm that there is a checkmark in
the Enable attachment rule checkbox
on the Mail Attachment Rule dialog
box (figure 87). You have three options in the Apply action to messages containing attachments with one of these
properties frame:
Attachment name
Select this option and type in a name for the attachment,
including file name and file extension, in the text box next to this option.
Use this option if you don’t want to block all attachments with a particular
file extension, but you do want to block a specific file name. For example, you
do not want to block all .zip files, but you do want to block a file named exploit.zip.
Attachment extension
It is more common to block all files with a specific file
extension. For example, if you want to block all files with the exe file extension, select this option
and then type in either exe or .exe in the text box to the right of
this option.
Attachment size limit
(in bytes)
You can also block attachments based on their size. Select
this option and type in the size of the file extension you want to block.
Click the down arrow for the Action drop down list box. You have the following options:
Delete message
The SMTP message is deleted without
being saved or informing anyone that it has been deleted.
Hold Message
The SMTP message is held in the BADMAIL directory in the SMTP service’s
folder hierarchy. You can view components of the held message, but the message is not saved in a format that you can easily forward to the
recipient.
Forward message to
The SMTP message is forwarded to an
email address you configure in this rule. Each rule can have a different email
address that the message is forwarded to.
In this example we’ll select the Forward message to option so that you can see how to enter the
forwarding address.
Figure 87

9.
When you select the Forward message to option, a text box
appears that allows you to enter an email address to forward the message to.
However, the server must be able to resolve the address of the mail domain of
this user.
For example, in figure 88 we have entered the email address smtpsecurityadmin@internal.net.
The ISA Server firewall must be able to access an MX record for the
internal.net domain. The ISA Server firewall forwards the message to the SMTP
server responsible for the internal.net mail based on the information in the MX
record.
In this example the firewall is configured
with a DNS server address of a DNS server on the internal network that can
resolve both internal and external network names. The message is forwarded to the internal address of the Exchange server.
You must configure a split DNS
infrastructure if the internal.net
domain is available to both internal and external users.
Note:
Please refer to ISA Server 2000 Exchange
Server 2000/2003 Deployment Kit document Configuring DNS to Support
Exchange Server Publishing for information on how to create a
split DNS to support SMTP server publishing.
Click OK in the Mail Attachment Rule dialog box. Click on the SMTP Commands
tab.
Figure 88

10. The settings on the SMTP Commands tab are
mediated by the SMTP filter component. The SMTP Message Screener does
not evaluate SMTP commands and it does not protect against buffer overflow conditions.
The commands in the list are limited to a pre-defined length. If an incoming
SMTP connection sends a command that exceeds the length allowed, then the
connection is dropped. In addition, if a command is sent over the SMTP channel is not on this list, it is
dropped.
Click the Add
button to add an SMTP command to the list (figure 89).
Figure 89

11. A command you may want to enter into
the list of allowed SMTP commands is the AUTH
command. This is required if you want to allow external users to authenticate
with an SMTP server that is published via an SMTP Server Publishing Rule. Users
will not be able to authenticate with a SMTP server Published via an SMTP
Server Publishing Rule if the AUTH command is not
added to the list and the SMTP filter is enabled.
Confirm that the Enable
an SMTP command checkbox is checked. Type AUTH in the Command Name text box. Type 1024 in the Maximum Length Bytes text box. Click OK in the SMTP Command Rule
dialog box (figure 90).
Figure 90

12. The new command appears in the list
of SMTP commands on the SMTP Commands
tab (figure 91). Click Apply and
then click OK.
Figure 91

13. Close the ISA Server Management console.
The ISA
Server firewall/SMTP server is now ready to filter SMTP messages based on the
parameters you set for the SMTP filter and SMTP Message Screener.
Configure the SMTP Server Publishing
Rule on the ISA Server Firewall
You can now
create your SMTP Server Publishing Rules.
1.
Open the ISA Management console, expand the Servers and Arrays node and expand the server name. Expand the Publishing node and click on the Server Publishing Rules node. Right
click on the Server Publishing Rules
node, point to New and click on Rule (figure 92).
Figure 92

2.
Type a name for the Sever Publishing
Rule in the Server publishing rule name
text box on the Welcome to the New
Server Publishing Rule Wizard page (figure 93). Click Next.
Figure 93

3.
Type in the IP address used by the
secure authenticating SMTP virtual server in the IP address of internal server text box on the Address Mapping page (figure 94). Click the Browse button under the External
IP address on the ISA server and select the IP address you want to use on
the external interface of the ISA Server firewall to accept incoming connection
requests to the secure authentication SMTP virtual server. Click OK after selecting the address in the New Sever Publishing Rule Wizard dialog
box.
Figure 94

4.
Click Next on the Address Mapping
page after the external IP address has been entered (figure 95).
Figure 95

5.
On the Protocol Settings page (figure 96), click the down arrow from the Apply the rule to this protocol drop
down list box and select the SMTP Server
Protocol Definition. Click Next.
Figure 96

6.
On the Client Type page, select the Any request option
and click Next (figure 97).
Figure 97

7.
Review the settings on the Complete the New Server Publishing Rule
Wizard page (figure 98), and click Finish.
Figure 98

Configure the Public DNS to Forward
Mail to Your Domains
Internet
SMTP servers and your remote users must be able to access the published SMTP
server using a fully qualified domain name. In addition, if you have users who
move between the internal and external network, you should consider using the
same domain name for internal and external access. You may not wish to host
your publicly available resources using your private domain name, but you
should reserve that domain name and configure a public DNS server to resolve
Internet accessible resources for your users.
The key to
making this work is to configure a split DNS infrastructure. The split DNS
allows your roving users to access mail resources
while on the road, or on the internal network, and never need to change mail
client or browser settings. Please refer to ISA Server 2000 Exchange Server 2000/2003 Deployment Kit document Configuring DNS to Support
Exchange Server Publishing