A Real Technical Challenge for Experts - Windows SBS 2003 Server Perfo

[kbsantosh1]

Hello Everybody,

I have a Windows SBS 2003, which is primarily used as a file server. Shared folders with plain ascii text files are accessed by users to read and write through a DOS 16 bit application. Users also use the shared folder to view "images" in the DOS application.

My challenge is

(a) The Page/Sec is peaking to 100% every 10 seconds and the result is erratic performance - delays experienced by user.

(B) The disk queue also peaks to 70-100% once in 10 seconds.

The server is a Dell PowerEdge 2800 with 4 GB RAM and a Raid 5 System. Approx 30-50 users connect to the Server at a time. The primary is a PDC for the domain. All possible services are stopped on this server and DNS, WINS etc is load balanced to a secondary server.

Any help to improve the performance of the server will be sincerely appreciated. Please let me know if I can offer any more stats to diagonize this further.

Thanks.

- Santy Balan

san.rely@gmail.com

====================================================================================================

===============

Here are few statistics of the Server :-

TOtal Memory : 4 GIG

Raid 5

C Drive : 12 GB {paging file : 2 - 2 GB}

D Drve : 240 GB {paging file : 8 - 20 GB}

Acting as File Server for shared folders. Accessed by 30-50 users to read/write text files into the shared fodler.

Pages per sec... peaks to 100% about every 10 seconds

Average Disk Queue peaks to 70-80% about every 10 seconds

Process Stats

% of User Time = 5-10%

% of Processor Time = 100%

% of Privileged Time = 100%

Memory

Available Bytes = 100%

Free System Page Table Entries = 100%

Commit Limit = 100%

Page Reads = Spikes in 10 sec upto 100%

Page writes < 5%

Transition faults/sec = spikes in 10 sec upto 100%

Physical Disk

Average Disk Queue = Spikes in 10 sec upto 100%

Paging File

% of paying file usage < 1 % constant

Commit Charge (K)

Total : 2490632

Limit : 14409488

Peak : 2544516

Physical Memory (K)

Total : 4193360

Available : 1663572

System Cache : 1854808

Kernel Memory (K)

Total : 124652

Paged : 73828

NonPaged : 50824

Totals :

Handles : 25707

Threads : 1090

Processes : 74

CPU Usage = 2%

Commit Charge : 2431M/14071M

PF Usage : 2.37 GB

[cluberti]

You've posted this before .

Download perfwiz from the MS download site and run it in "high CPU" mode from another machine - note not to choose either the Exchange or Terminal Server option. You'll need to configure the performance logs and alerts service on the remote machine to log on with a domain admin account, or a local admin account on the server with the issue.

Once you've started perfwiz on the remote machine, give it 5 minutes, then stop the log, open it in perfmon, and check the following:

Memory:

- Pages/sec

- Page Faults/sec

- Page Reads/sec

- Page Writes/sec

- Pages Input/sec

- Pages Output/sec

Process

***For each process in the right-hand pane (NOT total!), select the following counters***

- Handle Count

- IO Read Bytes/sec

- IO Read Operations/sec

- IO Write Bytes/sec

- IO Write Operations/sec

- Page Faults/sec

- Private Bytes

- Thread Count

- Virtual Bytes

- Working Set

Also, run filemon (http://www.sysinternals.com) to see if you have any spikes to a particular file or set of files in this 10-second interval that can be seen.

Edited February 13, 2006 by cluberti

[fizban2]

now when you it is primarly being a file server. what else do this server actually do? does it act as a DC also or any other task that may add a preformance hit?

[kbsantosh1]

This is an AD and has WINS/DNS and other general services running on it. Point to note is it servers not more than 30 users at a time... and the CPU (double Xeon 4 GB RAM) usage is never more than 10%.

I am troubled by the Pages Per Sec and Average Disk Queue (RAID 5 trouble ?).

Before migrating to this current configuration, we had an old PowerEdge 1400 with 256 RAM and a single SCSI 12 GB. Users felt much better at the old server.. than with this Dell Monster 2800 4 GB Ram, 270 GB RAID 5.

The applications that the users are using is very simple 16 bit DOS program, which reads/writes plain text/ascii files to the server shared directory. It also reads Images from the server (70 k per image)

Why is the Paging / Disk Queue so high ? What can I do to analyze this and fix it ? Calling Dell does not help.

[nmX.Memnoch]

now when you it is primarly being a file server. what else do this server actually do? does it act as a DC also or any other task that may add a preformance hit?

It's SBS 2003 so in all likelihood it's running Active Directory, DNS (for AD), Exchange, SQL Server, IIS, Print services, etc. This still shouldn't be a problem for the level of hardware he's talking about and only 30 users.

Here's my reply from the previous thread regarding this same subject. In my opinion, the key words in his original post are "DOS 16-bit application".

The drives are configured as one large RAID5 array? Dell has a habit of doing this unless you specify otherwise. I generally use two drives in RAID1 for the OS and seperate the data onto another RAID5 array. Another thing I usually do is get the split backplane option so the OS drives are on one channel of the controller, while the data files are on another channel.

Still, 30-50 users shouldn't be causing this kind of performance. Do you have the RAID controller configured for Direct I/O or Cached I/O?

cluberti also brings up a good point about AV software.

Also, check for firmware updates for the RAID controller. Dell is pretty good about fixing firmware issues fairly quickly. There may also be a compatibility issue between the actual drives and the RAID controller...something that may not have been noticed by Dell until after you got your server. If you do the RAID controller firmware be sure to read the instructions before applying it. They usually require that you update the driver before updating the firmware, otherwise you could end up with a system that won't boot because the old driver doesn't recognize the new firmware. Again, read the instructions.

It probably also wouldn't hurt to check for System BIOS updates as well as the Embedded Server Management (ESM) firmware. The ESM will update the drive backplane firmware as well.

[kbsantosh1]

Is there any tool (may be Open Manage ??) or something... which will check and report that these wierd things of RAID is working properly or needs fix ?

I have some plain ascii text files which are used for lookup in the dos applications which are kept in the server shared directory.... does this lookup effect the performance ? as it has to read through the ascii file in seuential access ?

I also have images (70-80k) which is fetched from the shared directory for every line of ascii text keyed in.

With the kind of hardware the asci writes are very very meagre... but the reading of index files (flat files) and fetching of images.. could be some clues ???

Can I configure the SBS 2003 to make the Index files be cached at the user pc.. so that the index lookup happens at the user pc instead of the Server ?

What are the other ways I can bring down the Pages per Sec and Average Disk Queue ?

Any help is appreciated.

Thanx.

- Santy Balan