*
Microsoft.com Home|Site Map
Microsoft TechNet*
Search Microsoft.com for:
|TechNet Home
Commercial Internet System
Community
Downloads
Internet Explorer
Internet Information Server 3.0
Internet Information Server 4.0
Interoperability and Migration
IT Solutions
IT Tasks
Microsoft Mail
MS-DOS
Office
Personal Web Server
Project 98
Proxy Server
Security
SNA Server
Systems Management Server
Transaction Server
Visio
Windows 95
Windows 98
Windows 2000 Server
Windows for Workgroups
Windows ME
Windows NT Embedded
Windows NT Server 4.0
Windows NT Terminal Server
Windows NT Workstation

MS Site Server 3.0 Direct Mail

Capacity and Performance Analysis

April 1999

Microsoft Corporation

*
On This Page
Definition of TermsDefinition of Terms
Chapter 1Chapter 1
Chapter 2Chapter 2
Appendix AAppendix A
Appendix BAppendix B

Definition of Terms

Mailer - Specific TermsMeaning

Distribution List

List of recipients by e-mail name

Membership Directory

Directory containing user properties including e-mail name and interests

AUO

Active User Object: Web server script object that handles member operations in Site Server 3.0

ASP

Active Server Page: a page that contains BASIC-type code that executes on the Microsoft Web Server

Chapter 1

Overview

This document evaluates the performance and scalability characteristics of Microsoft® Site Server version 3.0 Direct Mail. Also demonstrated are procedures for identifying these characteristics. This document can be used to assess the value of adding resources, and to identify which resources would satisfy greater capacity needs.

Analyzing Direct Mail

System Configuration

In the test scenarios, five computers were used in the following configurations:

Direct Mail Processor Scaling

Sender:

CPU:

2 x 100-MHz Pentium

 

Memory:

128 MB of RAM

 

Disk:

4.3-GB SCSI

 

Network:

100 BaseT (switched)

 

Software:

Microsoft® Windows NT® version 4.0 Option Pack, Site Server version 3.0 (List Builder Service, Message Builder Service, and Direct Mailer)

SMTP:

CPU:

4 x 66-MHz Pentium

 

Memory:

512 MB of RAM

 

Disk:

2-GB, 2-GB, 20-GB SCSI

 

Network:

100 BaseT (switched)

 

Software:

Windows NT 4.0 Option Pack,
Windows NT Resource Kit

LDAP:

CPU:

2 x 200-MHz Pentium Pro

 

Memory:

256 MB of RAM

 

Disk:

4.3-GB SCSI

 

Network:

100 BaseT (switched)

 

Software:

Windows NT 4.0 Option Pack, Site Server 3.0

SQL:

CPU:

4 x 133-MHz Pentium

 

Memory:

512 MB of RAM

 

Disk:

22-GB SCSI RAID 0

 

Network:

100 BaseT (switched)

 

Software:

Windows NT 4.0 Option Pack
Microsoft® SQL Server™ version 6.5 Service Pack 3

Web Server:

CPU:

4 x 133-MHz Pentium

 

Memory:

1-GB of RAM

 

Disk:

12-GB SCSI RAID 0

 

Network:

100 BaseT (switched)

 

Software:

Windows NT 4.0 Option Pack
Microsoft® Internet Information Server version 4.0

Test Profiles

Test NameTest Description

Personalized Static Mail

Static HTML that is personalized on the Web Server

Non-Personalized Static Mail

Static HTML.

Personalized Dynamic

Mail generated via ASP using AUO and is also personalized on the Web server

Non-Personalized Mail

Mail generated via ASP using AUO.

Direct Mail Description

Introduction to Direct Mailer

Direct Mail sends electronic mail to groups of users. In addition to sending bulk mail to existing customers, you can use Direct Mail to send bulk e-mail to Web site users. You can also personalize messages according to these factors:

What parts of the site the user has visited (as recorded by usage logs).

Information provided by the user, such as personal preferences, products already purchased, and so on.

The number of times the user has browsed your site.

The mail process can be automated. For example, every Monday morning, Direct Mail can check which users have browsed the site for your new product in the past week. E-mail can then be sent to those users. Note that users who have never seen the site before will receive an introductory message; users who have been to the site more than once will receive more detailed information about the product. This task can be scheduled for low activity periods, such as before 4:00 A.M., so that your computers are free for your normal business activities.

Direct Mail utilizes several components:

Direct Mailer. Sets up mailings.

Microsoft Windows NT Scheduler Service. Determines when to send mailings.

Site Server List Builder Service. Sends queries to Microsoft® Site Server version 3.0 Analysis for mailing lists.

Site Server Analysis. Analyzes Web site usage, associating usage with user profiles imported from the Membership Directory.

Message Builder Service. Composes messages and sends them to a Simple Mail Transfer Protocol (SMTP) server.

Interaction and process of Direct Mail

The following diagram shows the Direct Mail service broken down by the physical servers that house the interacting components.

The process of sending mail operates in three distinct phases:

Distribution List Resolution. The Message Builder Service steps through the list and performs an LDAP search query to retrieve the e-mail name from the Membership Directory.

Mail Address Sort. The Message Builder Service sorts the list to remove duplicates.

Send and Receive:

Non-personalized static HTML. The HTML document is retrieved from the Web server and sent to the SMTP server for each e-mail recipient.

Non-personalized server side scripting (ASP). The ASP script is executed from the Web server, personalized for the first user in the distribution list, and then sent to the SMTP server for each e-mail recipient.

Personalized static HTML. For each recipient, the HTML document is retrieved from the Web server and then sent to the SMTP server.

Personalized server side scripting (ASP). The ASP script is executed and personalized for each e-mail recipient and then sent to the SMTP server.

Summary of Scalability and Performance

Based on the data collected in this document, the following assertions can be made about scaling and performance for Direct Mail:

Direct Mail scales to two processors, because the distribution list expansion code has 2 threads. A four-processor system doesn't substantially increase throughput.

Following the performance tuning procedures for the Membership Directory specifically for mail interaction will substantially reduce the time for the distribution list expansion. These procedures are described in the Site Server Service Pack 1 Readme.

Personalized mail takes longer than non-personalized mail for static mailings by approximately 40 percent and for dynamic mailings by approximately 140 percent due to Web server interaction.

Mail sent to a dynamic mailing list (dynamic mail) takes even more time and resources than static HTML-type mailings because the script must run on the web server to be processed before the mail can be sent. For example, in Non-personalized static versus dynamic mailing, the time to complete was between 9 and 22 percent faster.

Non-personalized Static Mailing was the fastest to complete and it also had the highest send rate at 54 per second.

Chapter 2

Detail Discussion of Scalability and Performance

Time to Complete a Mailing

This chart shows that the time resource cost for adding personalization to a simple HTML is approximately 40 percent greater.

It also shows that a simple dynamic script can increase the time to complete a mailing by 20 percent over a static script. The script in this example is in Appendix B. It is an uncomplicated script. More complex scripts will increase maximum throughput and time to completion.

Time per mailing by mail type

List SizePersonalized StaticNon-Personalized StaticPersonalized DynamicNon-Personalized Dynamic

1,000

1.07 minutes

0.76 minutes

not measured

2.16 minutes

10,000

28 minutes

10 minutes

25.25 minutes

12.25 minutes

50,000

61 minutes

44 minutes

119.5 minutes

49 minutes

100,000

127 minutes

89 minutes

240.25 minutes

97.75 minutes

Maximum transaction rates

These tables display the maximum mail send rate when it is in the send and receive mail phase. The rates show the maximum sustainable speed at different list sizes.

List size does not affect the maximum rates.

Non-personalized mail is faster since it does not have to go to the Web server.

Maximum send rate by mail type

List SizePersonalized StaticNon-personalized StaticPersonalized DynamicNon-personalized Dynamic

1,000

26 msg/sec

54 msg/sec

9.734 msg/sec

44 msg/sec

10,000

26 msg/sec

53.5 msg/sec

9.733 msg/sec

43.534 msg/sec

50,000

26 msg/sec

53.7 msg/sec

9.801 msg/sec

43.801 msg/sec

100,000

26 msg/sec

53.5 msg/sec

9.934 msg/sec

44.467 msg/sec

Memory Utilization on the Message Builder/List Builder Computer

This chart shows the maximum private bytes utilized by the Message Builder and List Builder Services while sending the different transactions. These memory consumption values are only for the mail services and do not include the memory used by Windows NT or any other process running on the server.

Memory utilization on Message Builder/List Builder by mail type

List SizePersonalized StaticNon-Personalized StaticPersonalized DynamicNon-Personalized Dynamic

1,000 Members

4,608 KB

4,476 KB

6,840 KB

6,733 KB

10,000 Members

9,332 KB

8,039 KB

9,056 KB

7,548 KB

50,000 Members

23,989 KB

22,624 KB

23,944 KB

21,372 KB

100,000 Members

40,916 KB

48,808 KB

47,509 KB

40,260 KB

SQL Resource Utilization

The SQL server maintains the database for the Membership Directory. During the first phase of the mailing operation it handles the requests from the LDAP server for the user objects. This table details the CPU utilization for this system. The disk subsystem on the SQL server is also utilized. The specifications and recommendations for this specific subsystem are documented in a separate paper. The SQL server should be optimized as described in the Site Server 3.0 Membership Directory Configuration and Tuning document, which is also available from Microsoft.

List SizeMeasured DataPersonalized StaticNon-Personalized StaticPersonalized DynamicNon-Personalized Dynamic

1,000

CPU

16%

16%

10%

11%

 

Phase Length

8 seconds

8 seconds

10 seconds

10 seconds

10,000

CPU

6.04%

5.86%

5.90%

6.34%

 

Phase Length

4 minutes

6 minutes

5 minutes

4.25 minutes

50,000

CPU

12.10%

11.02%

11.13%

11.01%

 

Phase Length

9 minutes

11 minutes

10 minutes

10.5 minutes

100,000

CPU

13.75%

13.44%

12.70%

11.60%

 

Phase Length

18 minutes

18 minutes

18.5 minutes

20.25 minutes

The phase length in this table refers to the active period for the SQL server specific to the mailer operation. According to the description above this would be phase one.

However, if the personalized script makes use of the AUO object then another request could be made through the LDAP server to the SQL server.

LDAP Server Resource Utilization

The LDAP Service handles the requests for name resolution during the first phase of the mailing operation. The Message Builder Service computer opens a single connection to the LDAP server and makes a continuous stream of requests. The table below details the resource utilization for the LDAP server in this phase.

List SizeMeasured DataPersonalized StaticNon-Personalized StaticPersonalized DynamicNon-Personalized Dynamic

1,000

LDAP Searches

16 req/sec

16 req/sec

9.367 req/sec

8 req/sec

 

CPU

17%

17%

12%

9%

 

Phase length

8 seconds

6 seconds

14 seconds

12 seconds

10,000

LDAP Searches

3.35 req/sec

6 req/sec

3.18 req/sec

3.2 req/sec

 

CPU

4.13%

9%

3.97%

4.08%

 

Phase length

4 minutes

13 minutes

5 minutes

4.25 minutes

50,000

LDAP Searches

7.58 req/sec

6.95 req/sec

8.14 req/sec

10.2 req/sec

 

CPU

7.71%

9.75%

8.70%

10.80%

 

Phase length

9 minutes

11 minutes

10 minutes

10.5 minutes

100,000

LDAP Searches

8.69 req/sec

7.7 req/sec

8.89 req/sec

13.1 req/sec

 

CPU

9.11%

14.20%

9.40%

13.80%

 

Phase length

18 minutes

20 minutes

18.5 minutes

20.25 minutes

The phase length in this table refers to the active period for the LDAP server specific to the mailer operation. According to the description above this would be phase one.

However, if the personalized script makes use of the AUO object, then another request could be made through the LDAP server to the SQL server.

WWW Resource Utilization

In this table, the second phase of the mail operation is detailed. The primary resource utilization for this server is the CPU. It should be noted that the script that is run will greatly affect the overall outcome of these results. The script that was used in this test is contained in Appendix B.

List SizeMeasured DataPersonalized StaticNon-Personalized StaticPersonalized DynamicNon-Personalized Dynamic

1,000

CPU

4%

1%

12%

1%

 

Phase Length

40 seconds

n/a

1 minute

n/a

10,000

CPU

8%

1%

13.31%

1%

 

Phase Length

6 minutes

n/a

17.5 minutes

n/a

50,000

CPU

7.80%

1%

11.91%

1%

 

Phase Length

33 minutes

n/a

68.5 minutes

n/a

100,000

CPU

7.65%

1%

12.70%

1%

 

Phase Length

64 minutes

n/a

179 minutes

n/a

SMTP Resource Utilization

The SMTP server was running an SMTP server emulator1 that did not actually send the message. Refer to the SMTP scaling documents available from Microsoft that address specifications for this component. The transaction rate generated by the mailer directly corresponds to the amount of mail traffic the server should handle.

Appendix A

Critical Monitoring Counters

All counters noted can be found in the Microsoft® Windows NT® Performance Monitor. These counters will be distributed among the machines in the Personalization and Membership (P&M) service group. The counters in the system and memory objects can be used to monitor capacity.

Physical Disk

Disk Writes/sec Disk activity should not sustain maximum transaction rate

Disk Reads/sec

System Object

Context switches/sec should be less than 15,000

%Total Processor should be less than 80%

Processor Object

%Processor Utilization (average) should be less than 80% (for each processor)

Memory Object

Available Bytes should be greater than 4 MB of RAM

Pages/sec should be less than 1 per second

Site Server Message Builder Service

Messages Sent/sec

SMTP Server

Messages Delivered/sec

Process Object

Private Bytes for the inetinfo, TMMLBSvc, TMsgBld processes

Appendix B

ASP Script for AUO

<%@LANGUAGE=VBScript%>
<%
     m_BindAsUserName = "cn=Administrator"
     m_BindAsPassword = "password"
%>
<!-- #INCLUDE FILE="dmheader.asp" -->
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
<html>
<head>
<meta name="GENERATOR" content="Microsoft FrontPage 3.0">
<!-- meta http-equiv="Content-Type" content="text/html"; charset="iso-8859-1" -->
<meta http-equiv="Content-Type" content="text/html">
<title>Direct Mailer Test - Simple AUO page</title>
</head>
<body>
<%
  response.write "Hello " & m_sFirstName & ", where do you want to go today?" & chr(13)
%>
</body>
</html>

HTML Page for Static Tests

<html>
<head>
<meta name="GENERATOR" content="Microsoft FrontPage 3.0">
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<title>Direct Mailer Test - Simple HTML Page</title>
</head>
<body>
Hello, World!
</body>
</html>

1 The program is a tool called SuperSink.exe.



© 2005 Microsoft Corporation. All rights reserved. Terms of Use |Trademarks |Privacy Statement
Microsoft