Jump to content

0x8000005 and 0xa000000


Recommended Posts

Just wondering if anyone has any ideas on this problem. A SQL job runs and it freezes up the whole system. We have to switch it off manually and looking at the event viewer we see 0x8000005 and odd 0xa000000 messages listed.

It an HP system with 32 bit op system but running on 64 bit harware (8GB ram).

Any ideas on resolving this? or pointing towards the right direction?

Thanks

Link to comment
Share on other sites


We have to switch it off manually and looking at the event viewer we see 0x8000005 and odd 0xa000000 messages listed.

You're seeing these because the machine wasn't shutdown properly. It doesn't give any useful infos.

Just wondering if anyone has any ideas on this problem. A SQL job runs and it freezes up the whole system.

Hard to say what happens with so little information. Does it freeze hard? (as in the whole server) -- if so, enable memory dumps so we can try to analyze them. Or is it just MS SQL that stops processing queries? (could be high load simply, you looked at CPU load/performance counters?)

What's the actual query and table schema? How many rows in the table? (very useful to know, so we can guess just how resource intensive to process, and even try to duplicate it) What version of MS SQL? Is the box fully patched & using the latest drivers?

We just can't solve the problem or guess what is it based on so little information unfortunately.

Link to comment
Share on other sites

We have to switch it off manually and looking at the event viewer we see 0x8000005 and odd 0xa000000 messages listed.

You're seeing these because the machine wasn't shutdown properly. It doesn't give any useful infos.

Just wondering if anyone has any ideas on this problem. A SQL job runs and it freezes up the whole system.

Hard to say what happens with so little information. Does it freeze hard? (as in the whole server) -- if so, enable memory dumps so we can try to analyze them. Or is it just MS SQL that stops processing queries? (could be high load simply, you looked at CPU load/performance counters?)

What's the actual query and table schema? How many rows in the table? (very useful to know, so we can guess just how resource intensive to process, and even try to duplicate it) What version of MS SQL? Is the box fully patched & using the latest drivers?

We just can't solve the problem or guess what is it based on so little information unfortunately.

Yes it freezes hard but was about to use Cluberti's method to get a memory dump

(Memory dump from an application/process that is HANGING (not crashing):

1. Create a directory called c:\adplus

2. Open a command prompt and change to the directory where you installed the debugging tools. By default, this is "C:\Program Files\Debugging Tools for Windows"

3. Type the following command in the command prompt:

cscript adplus.vbs -hang -pn appname -quiet -o c:\adplus (where "appname" is the .exe name of the application that is hanging)

4. Once the debugger has finished (this can take some time), the command prompt window(s) will close, and you will have data in your C:\adplus folder that can be analyzed.)

It may be a SQL issue as looking at the CPU usage it consumes almost half the memory and then suddenly the CPU has no activity....Box is almost fully patched i.e. it doesnt have the latest SQL 2005 SP2 update (it does has SP2 for SQL 2005 though). Ill get the table info and post if the memory dumps arent good enough.

Thanks

Link to comment
Share on other sites

Yes it freezes hard but was about to use Cluberti's method to get a memory dump

Then you're saying it's just SQL Server freezing, not Windows (which is what I meant). If windows was frozen, you wouldn't be able to use his method to get dumps...

If it's just SQL Server that stops responding, most likely it's just a query you're running that's too intensive, and it's just very busy trying to execute it. Some bad queries will tend to "freeze" a lot of different RDBMS'es (make them unresponsive for a while). Check your performance counters, and see what's happening (CPU pegged? I/O bottleneck? paging memory a lot? etc)

I'm not sure who's writing the queries being ran on it, but some beginners will write pretty bad queries that will do this kind of stuff: table scans on large tables, lots of joins, selecting unnecessary columns, no/bad indexes, poor choices in terms of database schema and data types, etc resulting in poor execution plans, high resource usage and all that. This is why I was asking about that other stuff.

Edited by crahak
Link to comment
Share on other sites

Yes it freezes hard but was about to use Cluberti's method to get a memory dump

Then you're saying it's just SQL Server freezing, not Windows (which is what I meant). If windows was frozen, you wouldn't be able to use his method to get dumps...

Its a bit strange....the process sqlserver.exe has high memory consumption and eventually the CPU shows no activity. But i can minimise windows etc and eventually i cant do anything for around 10 mins so switch off the server. Now for some reason it ran fine without any problems but when we ran it again the same issue occured.

It could be the query but how could i definately say that it is the problem as it was created by another outside source??

Thanks

Link to comment
Share on other sites

the process sqlserver.exe has high memory consumption

That's normal. It tries to keep data in memory, because it's a LOT faster to return results from data in memory than having to read it from disk first.

and eventually the CPU shows no activity

Well, maybe the query finished running, or maybe it did hang indeed.

Now for some reason it ran fine without any problems but when we ran it again the same issue occured.

Ok, now my #1 priority question would be, what changed since then? Any new apps, or new versions of some app or such you're using (that could make bad queries), or changes you've made recently to anything? If it's only been doing that for a couple days, and that something was changed a couple days ago, it might be a good idea to investigate.

Something else that is very much worth looking into is SQL Server query logging (there should be plenty of details about that in BOL, technet or MSDN). That will log the name of the database the query is ran against, the user who ran it, the query itself, its execution time and some more helpful data. It would help figuring out what's going on for sure.

Link to comment
Share on other sites

Thanks ill look into that but is the SQL profiler the same? Nothings really changed except the query has had indexes added to speed up the process

also one of the logs that i have from running the dump tools (whether thsi makes a difference or not) follows:

 0 32	0 System Process  
Command Line:
0 32 4 System
Command Line:
0 32 344 smss.exe
Command Line: \SystemRoot\System32\smss.exe
0 32 392 csrss.exe
Command Line: C:\WINDOWS\system32\csrss.exe ObjectDirectory=\Windows SharedSection=1024,3072,512 Windows=On SubSystemType=Windows ServerDll=basesrv,1 ServerDll=winsrv:UserServerDllInitialization,3 ServerDll=winsrv:ConServerDllInitialization,2 ProfileControl=Off MaxRequestThreads=16
0 32 420 winlogon.exe
Command Line: winlogon.exe
0 32 468 services.exe Svcs: Eventlog,PlugPlay
Command Line: C:\WINDOWS\system32\services.exe
0 32 480 lsass.exe Svcs: HTTPFilter,Netlogon,PolicyAgent,ProtectedStorage,SamSs
Command Line: C:\WINDOWS\system32\lsass.exe
0 32 640 svchost.exe Svcs: DcomLaunch
Command Line: C:\WINDOWS\system32\svchost.exe -k DcomLaunch
0 32 708 svchost.exe Svcs: RpcSs
Command Line: C:\WINDOWS\system32\svchost.exe -k rpcss
0 32 800 Smc.exe Svcs: SmcService
Command Line: "C:\Program Files\Symantec\Symantec Endpoint Protection\Smc.exe"
0 32 816 svchost.exe Svcs: Dhcp,Dnscache
Command Line: C:\WINDOWS\system32\svchost.exe -k NetworkService
0 32 868 svchost.exe Svcs: Alerter,LmHosts,W32Time
Command Line: C:\WINDOWS\system32\svchost.exe -k LocalService
0 32 884 svchost.exe Svcs: AeLookupSvc,BITS,Browser,CryptSvc,dmserver,EventSystem,helpsvc,lanmanserver,lanmanworkstation,N
etman,Nla,RasMan,Schedule,seclogon,SENS,ShellHWDetection,TrkWks,winmgmt,wuauserv,WZCSVC
Command Line: C:\WINDOWS\System32\svchost.exe -k netsvcs
0 32 960 ccSvcHst.exe Svcs: ccEvtMgr,ccSetMgr
Command Line: "C:\Program Files\Common Files\Symantec Shared\ccSvcHst.exe" /h ccCommon
0 32 1436 spoolsv.exe Svcs: Spooler
Command Line: C:\WINDOWS\system32\spoolsv.exe
0 32 1464 msdtc.exe Svcs: MSDTC
Command Line: C:\WINDOWS\system32\msdtc.exe
0 32 1576 cissesrv.exe Svcs: Cissesrv
Command Line: "C:\Program Files\HP\Cissesrv\cissesrv.exe"
0 32 1588 cpqrcmc.exe Svcs: CpqRcmc
Command Line: C:\WINDOWS\system32\cpqrcmc.exe
0 32 1604 vcagent.exe Svcs: cpqvcagent
Command Line: C:\hp\hpsmh\data\cgi-bin\vcagent\vcagent.exe
0 32 1648 svchost.exe Svcs: ERSvc
Command Line: C:\WINDOWS\System32\svchost.exe -k WinErr
0 32 1788 inetinfo.exe Svcs: IISADMIN,MSFtpsvc
Command Line: C:\WINDOWS\system32\inetsrv\inetinfo.exe
0 32 1832 MsDtsSrvr.exe Svcs: MsDtsServer
Command Line: "C:\Program Files\Microsoft SQL Server\90\DTS\Binn\MsDtsSrvr.exe"
0 32 1992 msftesql.exe Svcs: msftesql$SERVERNAME
Command Line: "C:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\Binn\msftesql.exe" -s:MSSQL.1 -f:SERVERNAME
0 32 2024 msmdsrv.exe Svcs: MSOLAP$SERVERNAME
Command Line: "C:\Program Files\Microsoft SQL Server\MSSQL.2\OLAP\bin\msmdsrv.exe" -s "C:\Program Files\Microsoft SQL Server\MSSQL.2\OLAP\Config"
0 32 2164 sqlservr.exe Svcs: MSSQL$BKUPEXEC
Command Line: "C:\Program Files\Microsoft SQL Server\MSSQL.4\MSSQL\Binn\sqlservr.exe" -sBKUPEXEC
0 32 2244 sqlservr.exe Svcs: MSSQL$SERVERNAME
Command Line: "C:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\Binn\sqlservr.exe" -sSERVERNAME
0 32 2288 svchost.exe Svcs: RemoteRegistry
Command Line: C:\WINDOWS\system32\svchost.exe -k regsvc
0 32 2304 ReportingServicesService.exe Svcs: ReportServer$SERVERNAME
Command Line: "C:\Program Files\Microsoft SQL Server\MSSQL.3\Reporting Services\ReportServer\bin\ReportingServicesService.exe"
0 32 2520 snmp.exe Svcs: SNMP
Command Line: C:\WINDOWS\System32\snmp.exe
0 32 2536 sqlbrowser.exe Svcs: SQLBrowser
Command Line: "C:\Program Files\Microsoft SQL Server\90\Shared\sqlbrowser.exe"
0 32 2588 sqlwriter.exe Svcs: SQLWriter
Command Line: "C:\Program Files\Microsoft SQL Server\90\Shared\sqlwriter.exe"
0 32 2640 Rtvscan.exe Svcs: Symantec AntiVirus
Command Line: "C:\Program Files\Symantec\Symantec Endpoint Protection\Rtvscan.exe"
0 32 2692 sysdown.exe Svcs: sysdown
Command Line: C:\WINDOWS\system32\sysdown.exe
0 32 2752 smhstart.exe Svcs: SysMgmtHp
Command Line: C:\hp\hpsmh\bin\smhstart.exe
0 32 2952 hpsmhd.exe
Command Line: C:\hp\hpsmh\bin\hpsmhd.exe -fC:/hp/hpsmh/conf/smhpd.conf
0 32 2972 beremote.exe Svcs: BackupExecAgentAccelerator
Command Line: "C:\Program Files\Symantec\Backup Exec\beremote.exe"
0 32 3216 rotatelogs.exe
Command Line: C:\hp\hpsmh\bin\rotatelogs.exe C:/hp/hpsmh/logs/error_log 5M
0 32 3224 rotatelogs.exe
Command Line: C:\hp\hpsmh\bin\rotatelogs.exe C:/hp/hpsmh/logs/access_log 5M
0 32 3244 pvlsvr.exe Svcs: BackupExecDeviceMediaService
Command Line: "C:\Program Files\Symantec\Backup Exec\pvlsvr.exe"
0 32 3256 hpsmhd.exe
Command Line: C:\hp\hpsmh\bin\hpsmhd.exe -d C:/hp/hpsmh -f C:/hp/hpsmh/conf/smhpd.conf
0 32 3304 rotatelogs.exe
Command Line: C:\hp\hpsmh\bin\rotatelogs.exe C:/hp/hpsmh/logs/error_log 5M
0 32 3312 rotatelogs.exe
Command Line: C:\hp\hpsmh\bin\rotatelogs.exe C:/hp/hpsmh/logs/access_log 5M
0 32 4168 cpqnimgt.exe Svcs: CpqNicMgmt
Command Line: C:\WINDOWS\system32\CPQNiMgt\cpqnimgt.exe
0 32 4200 cqmgserv.exe Svcs: CqMgServ
Command Line: C:\WINDOWS\system32\CpqMgmt\cqmgserv\cqmgserv.exe
0 32 4224 cqmgstor.exe Svcs: CqMgStor
Command Line: C:\WINDOWS\system32\CpqMgmt\cqmgstor\cqmgstor.exe
0 32 4248 SQLAGENT90.EXE Svcs: SQLAgent$SERVERNAME
Command Line: "C:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\Binn\SQLAGENT90.EXE" -i SERVERNAME
0 32 4340 svchost.exe Svcs: W3SVC
Command Line: C:\WINDOWS\System32\svchost.exe -k iissvcs
0 32 4632 cqmghost.exe Svcs: CqMgHost
Command Line: C:\WINDOWS\system32\CpqMgmt\cqmghost\cqmghost.exe
0 32 4672 wmiprvse.exe
Command Line: C:\WINDOWS\system32\wbem\wmiprvse.exe
0 32 4828 beserver.exe Svcs: BackupExecRPCService
Command Line: "C:\Program Files\Symantec\Backup Exec\beserver.exe"
0 32 5112 benetns.exe Svcs: BackupExecAgentBrowser
Command Line: "C:\Program Files\Symantec\Backup Exec\benetns.exe"
0 32 5192 bengine.exe Svcs: BackupExecJobEngine
Command Line: "C:\Program Files\Symantec\Backup Exec\bengine.exe"
0 32 5332 svchost.exe Svcs: TermService
Command Line: C:\WINDOWS\System32\svchost.exe -k termsvcs
0 32 6088 wmiprvse.exe
Command Line: C:\WINDOWS\system32\wbem\wmiprvse.exe
0 32 6000 svchost.exe Svcs: TapiSrv
Command Line: C:\WINDOWS\System32\svchost.exe -k tapisrv
2 32 6352 csrss.exe Title:
Command Line: C:\WINDOWS\system32\csrss.exe ObjectDirectory=\Windows SharedSection=1024,3072,512 Windows=On SubSystemType=Windows ServerDll=basesrv,1 ServerDll=winsrv:UserServerDllInitialization,3 ServerDll=winsrv:ConServerDllInitialization,2 ProfileControl=Off MaxRequestThreads=16
2 32 6376 winlogon.exe
Command Line: winlogon.exe
2 32 6652 rdpclip.exe Title: RDPClip - Receive Thread
Command Line: rdpclip
2 32 6732 explorer.exe Title: Program Manager
Command Line: C:\WINDOWS\Explorer.EXE
2 32 6864 ccApp.exe Title: CCALERT
Command Line: "C:\Program Files\Common Files\Symantec Shared\ccApp.exe"
2 32 6872 SmcGui.exe Title: Network Activity
Command Line: \\.\pipe\SygateSecurityAgentR6334T202109 \\.\pipe\SygateSecurityAgentW26500T202109
2 32 6896 ctfmon.exe Title:
Command Line: "C:\WINDOWS\system32\ctfmon.exe"
0 32 2492 logon.scr
Command Line: logon.scr /s
2 32 2056 cmd.exe Title: C:\WINDOWS\system32\cmd.exe - cscript adplus.vbs -hang -pn sqlservr.exe -quiet -o c:\MDUMP
Command Line: "C:\WINDOWS\system32\cmd.exe"
2 32 7944 taskmgr.exe Title: Windows Task Manager
Command Line: taskmgr.exe
2 32 7896 SqlWb.exe Title: Microsoft SQL Server Management Studio
Command Line: "C:\Program Files\Microsoft SQL Server\90\Tools\Binn\VSShell\Common7\IDE\SqlWb.exe"
0 32 5680 dllhost.exe Svcs: COMSysApp Mts: System Application
Command Line: C:\WINDOWS\system32\dllhost.exe /Processid:{02D4B3F1-FD88-11D1-960D-00805FC79235}
0 32 2284 DTExec.exe
Command Line: /FILE "\\SVROne\SSIS.dtsx" /MAXCONCURRENT " -1 " /CHECKPOINTING OFF /REPORTING E
2 32 6320 cscript.exe Title:
Command Line: cscript adplus.vbs -hang -pn sqlservr.exe -quiet -o c:\MDUMP
2 32 6308 cmd.exe Title: C:\WINDOWS\system32\cmd.exe
Command Line: "C:\WINDOWS\system32\cmd.exe" /c tlist.exe -v >"c:\MDUMP\Hang_Mode__Date_08-05-2008__Time_15-37-0707\Process_List.txt"
2 32 5240 tlist.exe
Command Line: tlist.exe -v

Edited by Bad boy Warrior
Link to comment
Share on other sites

80000005 = an error has occurred (not really useful) and 0xa000000 = what is logged when the next admin user logs on after a machine is forcefully powered off and chooses a reason (any reason) from the unclean shutdown dialog box.

Honestly, it'd be better if you could get a complete dump of the entire system the next time this occurs, because what we've got currently is not going to be enough to help you.

Link to comment
Share on other sites

When i created a full system dump i looked at the error codes and it seems all the error codes point to a manual dump being started (ifi understood it correctly):

----- 32 bit Kernel Full Dump Analysis

DUMP_HEADER32:

MajorVersion 0000000f

MinorVersion 00000ece

DirectoryTableBase 00039000

PfnDataBase 81400000

PsLoadedModuleList 808af9c8

PsActiveProcessHead 808b5be8

MachineImageType 0000014c

NumberProcessors 00000008

BugCheckCode 000000e2

BugCheckParameter1 00000000

BugCheckParameter2 00000000

BugCheckParameter3 00000000

BugCheckParameter4 00000000

PaeEnabled 00000000

KdDebuggerDataBlock 8089d3e0

Physical Memory Description:

Number of runs: 3

FileOffset Start Address Length

00001000 00001000 0009e000

0009f000 00100000 dfd56000

dfdf5000 dfe5e000 00001000

Last Page: dfdf5000 dfe5e000

KiProcessorBlock at 808aedc0

8 KiProcessorBlock entries:

ffdff120 f7727120 f772f120 f7737120 f773f120 f7747120 f774f120 f7757120

Windows XP Kernel Version 3790 (Service Pack 2) MP (8 procs) Free x86 compatible

Built by: 3790.srv03_sp2_gdr.070304-2240

Kernel base = 0x80800000 PsLoadedModuleList = 0x808af9c8

Debug session time: Fri Aug 08 09:06:29 2008

System Uptime: 0 days 0:12:27

start end module name

80800000 80a7a000 nt Checksum: FFFFFFFF Timestamp: unavailable (FFFFFFFE)

Unloaded modules:

b81d9000 b81ee000 NAVENG.SYS Timestamp: Fri Aug 08 08:55:24 2008 (489BFBEC)

b81ee000 b82be000 NAVEX15.SYS Timestamp: Fri Aug 08 08:55:24 2008 (489BFBEC)

b81d9000 b81ee000 NAVENG.SYS Timestamp: Fri Aug 08 08:55:23 2008 (489BFBEB)

b81ee000 b82be000 NAVEX15.SYS Timestamp: Fri Aug 08 08:55:23 2008 (489BFBEB)

ba0e0000 ba0f5000 NAVENG.SYS Timestamp: Fri Aug 08 08:55:22 2008 (489BFBEA)

ba0f5000 ba1c5000 NAVEX15.SYS Timestamp: Fri Aug 08 08:55:22 2008 (489BFBEA)

baf50000 baf5e000 imapi.sys Timestamp: Fri Aug 08 08:54:25 2008 (489BFBB1)

f7817000 f781f000 halfinch.sys Timestamp: Fri Aug 08 08:54:23 2008 (489BFBAF)

f780f000 f7817000 Sfloppy.SYS Timestamp: Fri Aug 08 08:54:23 2008 (489BFBAF)

f7607000 f7611000 Flpydisk.SYS Timestamp: Fri Aug 08 08:54:23 2008 (489BFBAF)

f74f7000 f7504000 WDFLDR.SYS Timestamp: Fri Aug 08 08:54:10 2008 (489BFBA2)

f723e000 f72a6000 bxvbdx.sys Timestamp: Fri Aug 08 08:54:10 2008 (489BFBA2)

Finished dump check

When reading the reason why it shutdown from eventviewer i now have:

Event Type: Error

Event Source: EventLog

Event Category: None

Event ID: 6008

Date: 13/08/2008

Time: 08:27:23

User: N/A

Computer: ELMI01

Description:

The previous system shutdown at 20:36:29 on 12/08/2008 was unexpected.

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

Data:

0000: 000807d8 000c0002 00240014 0051001d

0010: 000807d8 000c0002 00240013 0051001d

The data filed was in bytes but i clicked "words" to see if that makes it any helpful in resolving this. Or any other ideas if the log is of no help?

Thanks

Link to comment
Share on other sites

OK, you created a dump - can you put the .dmp file (compressed, of course) somewhere where it can be downloaded and analyzed?

I could but its around 3.5GB. Not sure what Winrar will compress this to unless theres any other info i can give?

Thanks

Probably down to around 500 or so MB.

Link to comment
Share on other sites

Windows XP Kernel Version 3790 (Service Pack 2) MP (8 procs) Free x86 compatible

Built by: 3790.srv03_sp2_gdr.070304-2240

Kernel base = 0x80800000 PsLoadedModuleList = 0x808af9c8

Debug session time: Fri Aug 08 09:06:29 2008

System Uptime: 0 days 0:12:27

Shooting from the hip time :)

You're running the W2K3 SP2 (GDR) kernel - I don't know if your system or SQL is particularly registry-intensive, but you may be experiencing the symptom from http://support.microsoft.com/kb/935926 : "A Windows Server 2003-based computer stops responding when the registry is in heavy use"

The latest (QFE) version of the kernel I could find (which includes the above hotfix and others) is available to the public here - http://support.microsoft.com/?id=954337 : "The computer could potentially have problems if DPC routines are executed at an IRQL that is greater than DISPATCH_LEVEL in Windows Server 2003"

Without a dump this is of course pure speculation ;)

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...