tmp007
Content Type
Profiles
Forums
Events
Posts posted by tmp007
-
-
Hello Mr Snrub,
Did you test uninstalling Symantec AV?
>> No I haven't tested that yet as I wanted to get to the root cause before attempt for any workaround..
Yes, I agree that the nonpaged pool is exhausted through allocations to "Irp "
Can you throw some light what exactly poinits to Symantec AV ?
The I/Os themselves are completed, but the pool allocations not freed, most likley due to some driver.
>> can we determine exactly which drivers?
0 -
New dump (with pool tagging enabled) available on
http://www.megaupload.com/?d=DYWZA3W7
Hope this helps to pin point the exact cause and solution.
(Plz explain in details on root cause as I am not that expert in understanding Windows stuff, it will help me :-))
0 -
Hello Cluberti and Mr Snrub,
Lets get back to the business :-)
1: kd> !poolused 7
Sorting by NonPaged Pool Consumed
Pool Used:
NonPaged Paged
Tag Allocs Frees Diff Used Allocs Frees Diff Used
Irp 1234650 923602 311048 193442080 0 0 0 0 Io, IRP packets
NDCM 4878844 4877733 1111 11535984 0 0 0 0 UNKNOWN pooltag 'NDCM', please update pooltag.txt
SpDN 117439 117420 19 9408720 602 602 0 0 UNKNOWN pooltag 'SpDN', please update pooltag.txt
Ar5k 295798 251837 43961 7129360 0 0 0 0 UNKNOWN pooltag 'Ar5k', please update pooltag.txt
MmCm 770 23 747 3625232 0 0 0 0 Calls made to MmAllocateContiguousMemory , Binary: nt!mm
tdLL 11832 9916 1916 1260968 0 0 0 0 UNKNOWN pooltag 'tdLL', please update pooltag.txt
Attv 1418997 1418635 362 744608 0 0 0 0 UNKNOWN pooltag 'Attv', please update pooltag.txt
Ddk 22 0 22 720968 84 78 6 336 Default for driver allocated memory (user's of ntddk.h)
Wdm 2955 2116 839 698352 298 287 11 1392 WDM
INTC 110935 110902 33 541608 347223 346967 256 10804344 Intel video driver
Thre 1260008 1259207 801 506232 0 0 0 0 Thread objects , Binary: nt!ps
File 7652723 7649529 3194 488176 0 0 0 0 File objects
Devi 966 446 520 365640 0 0 0 0 Device objects
AmlH 4 0 4 262144 0 0 0 0 ACPI AMLI Pooltags
Even 2354587 2349626 4961 242512 0 0 0 0 Event objects
SACM 54717 52659 2058 224912 0 0 0 0 UNKNOWN pooltag 'SACM', please update pooltag.txt
Mm 13 0 13 222432 641 637 4 2632 general Mm Allocations , Binary: nt!mm
CMpa 37521 33669 3852 215712 0 0 0 0 registry post apcs , Binary: nt!cm
Vad 681176 676726 4450 213600 0 0 0 0 Mm virtual address descriptors , Binary: nt!mm
NDpp 87 15 72 197840 0 0 0 0 packet pool , Binary: ndis.sys
Ntf0 3 0 3 196608 327432 326175 1257 95320 general pool allocation , Binary: ntfs.sys
usbp 4877 4827 50 193672 137 128 9 424 UNKNOWN pooltag 'usbp', please update pooltag.txt
Ntfr 28341 26017 2324 149192 0 0 0 0 ERESOURCE , Binary: ntfs.sys
AfdC 11124 10254 870 139200 0 0 0 0 Afd connection structure , Binary: afd.sys
Pool 4 1 3 135168 0 0 0 0 Pool tables, etc.
...
TOTAL 129490997 129081351 409646 236314128 226032011 225955018 76993 126236672
0 -
Hello Cluberti and Mr Snrub,
I am bit away from that machine at the moment, I'm pretty sure it have core'd till now.
I will upload it as soon as I get control over. Wont be much long :-)
0 -
My observation is that it crashes after 10 hours once I keep the machine up and running.
I have enabled the pool tagging and I will collect the dump and will let you know.
Thanks for your continual efforts.
0 -
Full Dump available here > http://www.megaupload.com/?d=VBI4U0LR
0 -
Thanks guys for considering this high priority.
1) Here is what I found : http://blogs.msdn.com/oldnewthing/archive/...04/9172708.aspx
2) Can you explain in details what the below means? curious to know what those number indicates too.
========
NonPagedPool Usage: 65534 ( 262136 Kb)
NonPagedPool Max: 65536 ( 262144 Kb)
********** Excessive NonPaged Pool Usage *****
===========
3) 1: kd> !poolused 7
unable to get PoolTrackTable - pool tagging is disabled, enable it to use this command
Use gflags.exe and check the box that says "Enable pool tagging".
4) Due to size limitation I have uploaded the "Mini070609-01.dmp"
Plz help me to pin pt the exact cause and way to fix it !
0 -
Thanks Mr Snrub !
Here is the output you requested.
1: kd> !vm
*** Virtual Memory Usage ***
Physical Memory: 521819 ( 2087276 Kb)
Page File: \??\C:\pagefile.sys
Current: 2095104 Kb Free Space: 2055696 Kb
Minimum: 2095104 Kb Maximum: 4190208 Kb
Available Pages: 213762 ( 855048 Kb)
ResAvail Pages: 423764 ( 1695056 Kb)
Locked IO Pages: 74 ( 296 Kb)
Free System PTEs: 119540 ( 478160 Kb)
Free NP PTEs: 0 ( 0 Kb)
Free Special NP: 0 ( 0 Kb)
Modified Pages: 484 ( 1936 Kb)
Modified PF Pages: 484 ( 1936 Kb)
NonPagedPool Usage: 65534 ( 262136 Kb)
NonPagedPool Max: 65536 ( 262144 Kb)
********** Excessive NonPaged Pool Usage *****
PagedPool 0 Usage: 28165 ( 112660 Kb)
PagedPool 1 Usage: 1713 ( 6852 Kb)
PagedPool 2 Usage: 1690 ( 6760 Kb)
PagedPool 3 Usage: 1682 ( 6728 Kb)
PagedPool 4 Usage: 1670 ( 6680 Kb)
PagedPool Usage: 34920 ( 139680 Kb)
PagedPool Maximum: 91136 ( 364544 Kb)
********** 19498 pool allocations have failed **********
Session Commit: 401 ( 1604 Kb)
Shared Commit: 5977 ( 23908 Kb)
Special Pool: 0 ( 0 Kb)
Shared Process: 6784 ( 27136 Kb)
PagedPool Commit: 34920 ( 139680 Kb)
Driver Commit: 3706 ( 14824 Kb)
Committed pages: 254596 ( 1018384 Kb)
Commit limit: 1006752 ( 4027008 Kb)
Total Private: 155603 ( 622412 Kb)
1474 firefox.exe 45953 ( 183812 Kb)
0a94 Rtvscan.exe 13436 ( 53744 Kb)
1134 wlmail.exe 9173 ( 36692 Kb)
0fcc explorer.exe 7287 ( 29148 Kb)
0bf8 java.exe 7269 ( 29076 Kb)
1e08 java.exe 6639 ( 26556 Kb)
0530 svchost.exe 4617 ( 18468 Kb)
1a68 issimgui.exe 4323 ( 17292 Kb)
1518 PCSuite.exe 3282 ( 13128 Kb)
03d4 winlogon.exe 2930 ( 11720 Kb)
1954 NclBCBTSrv.exe 2606 ( 10424 Kb)
03a8 issimsvc.exe 2079 ( 8316 Kb)
066c svchost.exe 1975 ( 7900 Kb)
0260 c4ebreg.exe 1958 ( 7832 Kb)
0844 hpqtra08.exe 1886 ( 7544 Kb)
00f8 svchost.exe 1711 ( 6844 Kb)
06a0 spoolsv.exe 1641 ( 6564 Kb)
0c84 BTSTAC~1.EXE 1635 ( 6540 Kb)
05fc SPBBCSvc.exe 1582 ( 6328 Kb)
0c34 YahooAUService. 1396 ( 5584 Kb)
0180 ccEvtMgr.exe 1392 ( 5568 Kb)
0320 ccProxy.exe 1270 ( 5080 Kb)
07c0 ccSetMgr.exe 1133 ( 4532 Kb)
0c7c AcSvc.exe 1083 ( 4332 Kb)
040c lsass.exe 1083 ( 4332 Kb)
03a4 ISSVC.exe 1059 ( 4236 Kb)
0850 acs.exe 1056 ( 4224 Kb)
00e8 svchost.exe 1018 ( 4072 Kb)
03ec wweb32.exe 999 ( 3996 Kb)
0750 cisvc.exe 981 ( 3924 Kb)
15bc artcore.exe 952 ( 3808 Kb)
0834 BTTray.exe 919 ( 3676 Kb)
0f78 VPTray.exe 901 ( 3604 Kb)
0b0c SymSPort.exe 899 ( 3596 Kb)
04d4 svchost.exe 829 ( 3316 Kb)
1e20 UEDIT32.EXE 768 ( 3072 Kb)
0508 svchost.exe 743 ( 2972 Kb)
09dc svchost.exe 714 ( 2856 Kb)
0d08 PCS_AGNT.EXE 709 ( 2836 Kb)
02fc isamtray.exe 707 ( 2828 Kb)
0a68 hpqste08.exe 660 ( 2640 Kb)
0570 svchost.exe 627 ( 2508 Kb)
083c svchost.exe 611 ( 2444 Kb)
0770 jqs.exe 589 ( 2356 Kb)
04e4 SNDSrvc.exe 580 ( 2320 Kb)
0734 svchost.exe 573 ( 2292 Kb)
0554 btwdins.exe 558 ( 2232 Kb)
0400 services.exe 537 ( 2148 Kb)
0728 AppleMobileDevi 514 ( 2056 Kb)
0798 DefWatch.exe 513 ( 2052 Kb)
03bc csrss.exe 508 ( 2032 Kb)
0978 SavRoam.exe 478 ( 1912 Kb)
1908 NclUSBSrv.exe 468 ( 1872 Kb)
0dfc SvcGuiHlpr.exe 444 ( 1776 Kb)
0114 svchost.exe 436 ( 1744 Kb)
0a0c cidaemon.exe 423 ( 1692 Kb)
0630 svchost.exe 415 ( 1660 Kb)
11b8 cmd.exe 406 ( 1624 Kb)
013c cmd.exe 406 ( 1624 Kb)
04fc AcPrfMgrSvc.exe 404 ( 1616 Kb)
0884 NetCfgSv.EXE 403 ( 1612 Kb)
1560 ServiceLayer.ex 354 ( 1416 Kb)
0774 CDSWinSrv.exe 330 ( 1320 Kb)
0950 svchost.exe 329 ( 1316 Kb)
05b8 alg.exe 325 ( 1300 Kb)
0bac wrtService.exe 214 ( 856 Kb)
1504 NclRSSrv.exe 188 ( 752 Kb)
1708 NclIrSrv.exe 169 ( 676 Kb)
0b50 TPHDEXLG.exe 162 ( 648 Kb)
04b4 ibmpmsvc.exe 150 ( 600 Kb)
1244 artifdown.exe 112 ( 448 Kb)
080c ntmulti.exe 74 ( 296 Kb)
038c smss.exe 42 ( 168 Kb)
0004 System 8 ( 32 Kb)
1e0c W32MAIN2.EXE 0 ( 0 Kb)
1dc0 W32MAIN2.EXE 0 ( 0 Kb)
1d90 W32MAIN2.EXE 0 ( 0 Kb)
1d30 W32MAIN2.EXE 0 ( 0 Kb)
1cc0 W32MAIN2.EXE 0 ( 0 Kb)
1aac W32MAIN2.EXE 0 ( 0 Kb)
1a98 W32MAIN2.EXE 0 ( 0 Kb)
1494 W32MAIN2.EXE 0 ( 0 Kb)
13b8 cmd.exe 0 ( 0 Kb)
1200 WINWORD.EXE 0 ( 0 Kb)
1168 W32MAIN2.EXE 0 ( 0 Kb)
0e40 pcssnd.exe 0 ( 0 Kb)
0618 cmd.exe 0 ( 0 Kb)
0528 W32MAIN2.EXE 0 ( 0 Kb)
Secondly. its not SP3. Its Windows XP Kernel Version 2600 (Service Pack 2) MP (2 procs) Free x86 compatible
0 -
Can we find out which process or application cause this crashed?
It appears that csrss.exe was terminated... but why?
Can we exactly pin-point the cause ?
0 -
Hello Windows Guru,
I am facing some nasty problem here
I would like to know root cause of the system crash - which system process/object terminates causing windows to crash.
I have opened the dump file (MEMORY_06Jul09_3-54PM.DMP) in windbg.
Below is the result....
======================================
Microsoft ® Windows Debugger Version 6.11.0001.404 X86
Copyright © Microsoft Corporation. All rights reserved.
Loading Dump File [C:\Documents and Settings\Administrator\Desktop\MEMORY_06Jul09_3-54PM\MEMORY_06Jul09_3-54PM.DMP]
Kernel Summary Dump File: Only kernel address space is available
Symbol search path is: SRV*c:\symbols*http://msdl.microsoft.com/download/symbols
Executable search path is:
Windows XP Kernel Version 2600 (Service Pack 2) MP (2 procs) Free x86 compatible
Product: WinNt, suite: TerminalServer SingleUserTS
Built by: 2600.xpsp_sp2_qfe.070227-2300
Machine Name:
Kernel base = 0x804d7000 PsLoadedModuleList = 0x805624a0
Debug session time: Mon Jul 6 15:52:59.656 2009 (GMT+10)
System Uptime: 0 days 19:41:57.373
Loading Kernel Symbols
...............................................................
................................................................
................................................................
....................
Loading User Symbols
PEB is paged out (Peb.Ldr = 7ffd800c). Type ".hh dbgerr001" for details
Loading unloaded module list
..................................
*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************
Use !analyze -v to get detailed debugging information.
BugCheck F4, {3, 88575da0, 88575f14, 80604528}
unable to get nt!KiCurrentEtwBufferOffset
unable to get nt!KiCurrentEtwBufferBase
PEB is paged out (Peb.Ldr = 7ffd800c). Type ".hh dbgerr001" for details
PEB is paged out (Peb.Ldr = 7ffd800c). Type ".hh dbgerr001" for details
Probably caused by : hardware_disk
Followup: MachineOwner
---------
1: kd> !analyze -v
*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************
CRITICAL_OBJECT_TERMINATION (f4)
A process or thread crucial to system operation has unexpectedly exited or been
terminated.
Several processes and threads are necessary for the operation of the
system; when they are terminated (for any reason), the system can no
longer function.
Arguments:
Arg1: 00000003, Process
Arg2: 88575da0, Terminating object
Arg3: 88575f14, Process image file name
Arg4: 80604528, Explanatory message (ascii)
Debugging Details:
------------------
unable to get nt!KiCurrentEtwBufferOffset
unable to get nt!KiCurrentEtwBufferBase
PEB is paged out (Peb.Ldr = 7ffd800c). Type ".hh dbgerr001" for details
PEB is paged out (Peb.Ldr = 7ffd800c). Type ".hh dbgerr001" for details
PROCESS_OBJECT: 88575da0
IMAGE_NAME: hardware_disk
DEBUG_FLR_IMAGE_TIMESTAMP: 0
FAULTING_MODULE: 00000000
PROCESS_NAME: csrss.exe
EXCEPTION_RECORD: a92be9d8 -- (.exr 0xffffffffa92be9d8)
ExceptionAddress: 75b7b399
ExceptionCode: c0000006 (In-page I/O error)
ExceptionFlags: 00000000
NumberParameters: 3
Parameter[0]: 00000000
Parameter[1]: 75b7b399
Parameter[2]: c000009a
Inpage operation failed at 75b7b399, due to I/O error c000009a
EXCEPTION_CODE: (NTSTATUS) 0xc0000006 - The instruction at "0x%08lx" referenced memory at "0x%08lx". The required data was not placed into memory because of an I/O error status of "0x%08lx".
DEFAULT_BUCKET_ID: DRIVER_FAULT
ERROR_CODE: (NTSTATUS) 0xc0000006 - The instruction at "0x%08lx" referenced memory at "0x%08lx". The required data was not placed into memory because of an I/O error status of "0x%08lx".
EXCEPTION_PARAMETER1: 00000000
EXCEPTION_PARAMETER2: 75b7b399
EXCEPTION_PARAMETER3: c000009a
IO_ERROR: (NTSTATUS) 0xc000009a - Insufficient system resources exist to complete the API.
EXCEPTION_STR: 0xc0000006_c000009a
FAULTING_IP:
+325952f0151dfdc
75b7b399 ?? ???
BUGCHECK_STR: 0xF4_IOERR_C000009A
STACK_TEXT:
a92be520 80634281 000000f4 00000003 88575da0 nt!KeBugCheckEx+0x1b
a92be544 806044e6 80604528 88575da0 88575f14 nt!PspCatchCriticalBreak+0x75
a92be574 804dd99f 88575fe8 c0000006 a92be9b0 nt!NtTerminateProcess+0x7d
a92be574 804e46a7 88575fe8 c0000006 a92be9b0 nt!KiFastCallEntry+0xfc
a92be5f4 80522128 ffffffff c0000006 a92be9f8 nt!ZwTerminateProcess+0x11
a92be9b0 80505460 a92be9d8 00000000 a92bed64 nt!KiDispatchException+0x3a0
a92bed34 804e12a8 0375fbe8 0375fc08 00000000 nt!KiRaiseException+0x175
a92bed50 804dd99f 0375fbe8 0375fc08 00000000 nt!NtRaiseException+0x33
a92bed50 75b7b399 0375fbe8 0375fc08 00000000 nt!KiFastCallEntry+0xfc
WARNING: Frame IP not in any known module. Following frames may be wrong.
0375fff4 00000000 00000000 00000000 00000000 0x75b7b399
STACK_COMMAND: kb
FOLLOWUP_IP:
+325952f0151dfdc
75b7b399 ?? ???
FOLLOWUP_NAME: MachineOwner
MODULE_NAME: hardware_disk
FAILURE_BUCKET_ID: 0xF4_IOERR_C000009A_IMAGE_hardware_disk
BUCKET_ID: 0xF4_IOERR_C000009A_IMAGE_hardware_disk
Followup: MachineOwner
---------
1: kd> !process ffffffff88575da0 3
PROCESS 88575da0 SessionId: 0 Cid: 03bc Peb: 7ffd8000 ParentCid: 038c
DirBase: 20fd0000 ObjectTable: e194ee90 HandleCount: 996.
Image: csrss.exe
VadRoot 87f59568 Vads 165 Clone 0 Private 413. Modified 6987. Locked 0.
DeviceMap e1008620
Token e53ec030
ElapsedTime 19:41:33.734
UserTime 00:00:04.390
KernelTime 00:00:16.640
QuotaPoolUsage[PagedPool] 115384
QuotaPoolUsage[NonPagedPool] 7512
Working Set Sizes (now,min,max) (1161, 50, 345) (4644KB, 200KB, 1380KB)
PeakWorkingSetSize 1343
VirtualSize 67 Mb
PeakVirtualSize 91 Mb
PageFaultCount 28724
MemoryPriority BACKGROUND
BasePriority 13
CommitCharge 508
THREAD 885d0da8 Cid 03bc.03c4 Teb: 7ffde000 Win32Thread: e5be0008 WAIT: (WrLpcReply) UserMode Non-Alertable
885d0f9c Semaphore Limit 0x1
THREAD 885d0b30 Cid 03bc.03c8 Teb: 7ffdd000 Win32Thread: e16dd7b8 WAIT: (UserRequest) UserMode Alertable
885cdde8 SynchronizationEvent
88644320 SynchronizationEvent
885cddb8 SynchronizationEvent
THREAD 885cc020 Cid 03bc.03cc Teb: 7ffdc000 Win32Thread: e5a8deb0 WAIT: (WrLpcReceive) UserMode Non-Alertable
89078c68 Semaphore Limit 0x7fffffff
THREAD 8904bb38 Cid 03bc.03d0 Teb: 7ffdb000 Win32Thread: 00000000 WAIT: (WrLpcReceive) UserMode Non-Alertable
890c0b60 Semaphore Limit 0x7fffffff
THREAD 8854e638 Cid 03bc.03dc Teb: 7ffda000 Win32Thread: e5bd69f8 WAIT: (WrLpcReceive) UserMode Non-Alertable
89078c68 Semaphore Limit 0x7fffffff
THREAD 8856fa20 Cid 03bc.03e0 Teb: 7ffd9000 Win32Thread: e123e598 WAIT: (WrUserRequest) KernelMode Alertable
88ffa418 SynchronizationEvent
885cd658 SynchronizationEvent
8905ee08 NotificationTimer
886270a0 SynchronizationEvent
80568420 NotificationEvent
885ce280 SynchronizationEvent
886445e8 SynchronizationTimer
THREAD 88571688 Cid 03bc.03e4 Teb: 7ffd7000 Win32Thread: e4bf4008 WAIT: (WrUserRequest) UserMode Non-Alertable
8856f3e0 SynchronizationEvent
8904f978 SynchronizationEvent
88ff69b0 SynchronizationEvent
THREAD 88529020 Cid 03bc.0414 Teb: 7ffd6000 Win32Thread: e14cca50 WAIT: (WrUserRequest) UserMode Non-Alertable
885543c8 SynchronizationEvent
88ff3b30 SynchronizationEvent
THREAD 88509020 Cid 03bc.057c Teb: 7ffd5000 Win32Thread: e1b0c0c8 WAIT: (WrLpcReceive) UserMode Non-Alertable
89078c68 Semaphore Limit 0x7fffffff
THREAD 8833b508 Cid 03bc.0154 Teb: 7ffd4000 Win32Thread: e175cc30 RUNNING on processor 1
THREAD 8833d788 Cid 03bc.01a4 Teb: 7ffaf000 Win32Thread: e174bc90 WAIT: (WrUserRequest) UserMode Non-Alertable
88342de0 SynchronizationEvent
THREAD 89d4a650 Cid 03bc.0318 Teb: 7ffae000 Win32Thread: 00000000 WAIT: (UserRequest) UserMode Non-Alertable
89d64f54 NotificationEvent
--------------
1: kd> .exr 0xffffffffa92be9d8
ExceptionAddress: 75b7b399
ExceptionCode: c0000006 (In-page I/O error)
ExceptionFlags: 00000000
NumberParameters: 3
Parameter[0]: 00000000
Parameter[1]: 75b7b399
Parameter[2]: c000009a
Inpage operation failed at 75b7b399, due to I/O error c000009a
=======================================================================
Please help me to find root cause of crash on ThinkPad Laptop.
Thanks for your help in advance.
0
For Windows gurus
in Windows XP
Posted
Hello folks,
I have a scenario where small blocksize write (to SAN Storage and Local Disk) is inffecient whereas large blocksize write is efficient.
I am looking for a tool/utility on WINDOWS which can show the throughput at different blocksize writes. I know I can use "dd" command after using "Cygwin" interface. But I would like to know if there is any direct tool/utility on WINDOWS which can help me here.
Cheers