V3Decompress Error - Folding@Home

[dnamechanic]

Hi Win98 Experts

I use the term "Experts" because I read through the forums and can appreciate the depth of understanding that contributors to this forum exhibit.

--------------------------------------------------------------------------------------------------------

A decompression problem with Win98SE has been observed in the Folding@Home (F@H) distributed computing project.

This problem occurs when Win98 is trying to decompress a large work unit (initial size approximately 4 MB).

Part of the process is that workunits are downloaded and automatically decompressed via BZIP2.

I was told by F@H technical that:

"The "v3Decompress" function (actually a macro) is in the Cosm package, which is an abstraction layer with an API intended to make it easier to write DC software which is OS-independent. Both the Folding@home client and all the cores use Cosm, to varying degrees. The failing function is one which is doing, in this case, BZIP2 decompression. We don't know why it fails with Win98, but since the problem only affects really big files, we suspect it is a memory allocation error of some sort."

I have verified that WinXP will take a copy of the same downloaded work unit and correctly decompress it, while Win98SE, after a wait of a few minutes, yields the following error message:

[20:49:50] - Files status OK

[20:52:39] Couldn't v3Decompress

[20:52:39] - Fatal: Could not decompress work unit data.

[20:52:39] Error: Could not open work file

[20:52:39]

[20:52:39] Folding@home Core Shutdown: FILE_IO_ERROR

The error is independent of hardware. I have verified that it occurs on both AMD and Intel systems with Win98SE, and does not occur on WinXP systems either with AMD or Intel hardware. Physical RAM is not an issue, all sytems had more than sufficient RAM (512MB or more). In fact, a WinXP system decompressed fine with 256 MB RAM.

Win98 will decompress similar files that are just slightly smaller in initial size (than the ~4MB size).

This problem is now widely recognized among participants of the F@H program. A Google search on the term "V3decompress" attests to the extent that the problem has been observed.

Wondering if anyone here at MSFN knows about this sort of problem with Win98, or knows about a potential fix for the problem?

Any suggestions would be appreciated.

Thanks

-dnamechanic

[LLXX]

The decompression code needs to be checked. It's probably causing a memory leak. Nothing wrong with the OS itself, it's just that XP is more tolerant of memory leaks.

4Mb is by no means a large compressed file...

Edited May 20, 2006 by LLXX

[dnamechanic]

Thanks for your response LLXX,

...It's probably causing a memory leak. 4Mb is by no means a large compressed file...

As I understand memory leaks, they occur gradually after loading programs and closing programs. The V3Decompress error will occur even after a fresh reboot in Win98SE. But, I have never seen it occur in WinXP, even after a system has been running, unbooted for weeks, during which time many different programs were loaded and closed.

Yes, I agree 4MB is not a large file to decompress. On one of my systems, Win98SE regularily unzips (with WinZip) much larger files.

...The decompression code needs to be checked...

This seems the most direct approach. Tho the technical people at F@H are reluctant to do this.

"...part of it is the slower hardware issue."

The reference to slow hardware above is probably that many systems running Win98 are older systems and not as fast, therefore they contribute to the program at lower rates.

The technical reps are probably are spread too thin, dealing with WinXP, MacOS, Linux, and the various problems that occur with the different types of F@H work units.

As a result, people like myself are moving away fron Win98SE to WinXP even though there are some advantages with Win98SE. Personally, I run Win98SE on new systems with fast hardware that I build myself, but many F@H contributors that use Win98SE are using older systems, and may located in countries all over the world. So, it may not be as feasable for them to upgrade the O/S.

-dnamechanic

[LLXX]

As I understand memory leaks, they occur gradually after loading programs and closing programs. The V3Decompress error will occur even after a fresh reboot in Win98SE. But, I have never seen it occur in WinXP, even after a system has been running, unbooted for weeks, during which time many different programs were loaded and closed.

A memory leak can happen very quickly if a program just allocates memory without freeing it. I'd guess that in one of the decompression loops memory is being allocated but not freed. The reason why WinXP can handle this is that it is still leaking memory, it's just more resistant to its effects (unused virtual memory eventually gets moved into the swap file, which just grows in size.)

This seems the most direct approach. Tho the technical people at F@H are reluctant to do this.

"...part of it is the slower hardware issue."

The reference to slow hardware above is probably that many systems running Win98 are older systems and not as fast, therefore they contribute to the program at lower rates.

Slow? I'm running 98se on a 4.17GHz P4, it flies on a fast CPU

[charles__]

You might try this app to see what is running the background.

http://www.sysinternals.com/Utilities/ProcessExplorer.html

Works great for me. You might have them kill unneeded program in the backround that are not needed. Do they have DMA set for the harddrive? Could also have them to check device manager in safe mode for ghost hardware.

[dnamechanic]

A memory leak can happen very quickly if a program just allocates memory without freeing it. I'd guess that in one of the decompression loops memory is being allocated but not freed.

I admit that I don't quite understand this. The process is automatic and there are several steps involved that I am unaware of but, generally appears that the FAH executable checks file integrity, allocates memory for the files then immediately begins decompression. If the decompression is successful the the scientific core computation begins.

I interpret your comment above to be saying that Win98 leaks allocated memory within seconds during the FAH decompression processs.

The reason why WinXP can handle this is that it is still leaking memory, it's just more resistant to its effects (unused virtual memory eventually gets moved into the swap file, which just grows in size.)

OK, given your logic above this seems quite reasonable.

Slow? I'm running 98se on a 4.17GHz P4, it flies on a fast CPU

That's a nice overclock, LLXX.

Yes, LLXX, I completely understand you. I'm also running Win98 on some overclocked systems quite fast, and I happen to like Win98 for several reasons. The quoted comment was from the FAH representative's point-of-view.

But, you and I are not the majority.

In general, the FAH contributors with the newer-faster computers have WinXP.

You might try this app to see what is running the background.

Works great for me.

Thanks Charles, Process Explorer looks like a neat little program. I haven't seen it before. I do have utilities that tell me everything that is running in Win98. One that I often use is called EM Task Manager. I may play around with yours, also.

You might have them kill unneeded program in the backround that are not needed. Do they have DMA set for the harddrive? Could also have them to check device manager in safe mode for ghost hardware.

Well Charles, "them" is me

And, also hundreds of other FAH contributors that use Win98.

I use DMA access for the hard drive, but with or without DMA the decompress problem occurs.

I assure you that Win98SE completely fails the V3Decompress when a file of the size I mentioned earlier comes through.

I have a hard drive with nothing on it except a stripped-down version of Win98SE and the small FAH executable with a few small support files. When booted up and presented with a test FAH "large work unit", this stripped-down system fails.

The problem is not a complete show-stopper for Win98. In practice what happens after the V3Decompress error occurs, is that The FAH server reassigns the Win98 system to another unit (a smaller unit), then progress continues.

The thing I don't like is this:

The large work units (the ones that fail on Win98) are processed more much efficiently, especially on machines with large and fast cache and fast RAM. Therefore, computers that can process the "large units" appear to contribute more to the FAH effort. This is why I will probably quit using Win98 for FAH work. I have WinXP on one computer now.

And, of course the effect of Win98 losing the "big ones" contributes to the FAH management as seeing that Win98 systems are slower. So it's kinda self-fulfilling prophesy.

Many contributors using Win98 may be unable to upgrade to WinXP.

-dnamechanic

[RJARRRPCGP]

FYI, Windows 98 support hasn't ended yet, AFAIK. I believe that it won't stop until late June.

[dnamechanic]

The decompression code needs to be checked. It's probably causing a memory leak. Nothing wrong with the OS itself, it's just that XP is more tolerant of memory leaks.

Assuming that memory leaks are the cause of the V3Decompression error:

Does anyone know if any of the Service Packs or Win98SE-ME hybrid versions offered in this forum would help with memory leaks?

If so, which ones?

And, should they be installed on a fresh Win98-SE install, or on Win98-SE with official updates (IE-Explorer) etc. already installed.

First, I would like to try any potential fixes on one system before I give up on win98-SE and install XP.

Thanks,

dnamechanic

[LLXX]

It's your folding app that's leaking the memory, not the OS.

[dnamechanic]

It's your folding app that's leaking the memory, not the OS.

Thanks LLXX, I appreciate your comment.

Yes, that may be true that the Folding app. is "leaking" memory.

But nevertheless, the Folding@Home app. performs flawlessly when run on WinXP.

I am looking for a fix, and I personally cannot change the Folding application.

This V3Decompress problem seems more like a memory allocation problem, because it happens within seconds of beginning to decompress a file, with no other programs running.

[dnamechanic]

It's your folding app that's leaking the memory, not the OS.

Hi LLXX,

What do you think about the following request?

--------------------------------------------------------------------------------

Would anyone with a robust Win98 system be willing to test a Folding@Home decompression application?

This request is an effort to determine if any version of Win98 can run particular Work Units issued by Folding@Home, distributed computing at Stanford University.

A particular type of Work Unit runs flawlessly on WinXP, but fails completely on all known Win98 systems.

The failure in Win98 is called the V3Decompress Error.

The executable files required are:

FAH502-Console.exe 248 KB

FAHCore_78.exe 2,268 KB

The above files, I could send, or you could get by a download from:

http://folding.stanford.edu/download.html

Download Version # 5.02

The above executables should be located in a directory named "Folding@Home"

When FAH502-Console.exe is executed, it places an entry in the registry, as follows:

HKEY_LOCALMACHINE\SOFTWARE\PANDEGROUP\Folding@Home

PANDEGROUP is the organization at Stanford University responsible

for the Folding@Home project.

One other step required to run this "problem" F@H Work Unit is to add a binary value for my "User ID" in the registry under:

HKEY_LOCALMACHINE\SOFTWARE\PANDEGROUP\Folding@Home

The "Value name": UserID

The binary "Value data": "My personal ID, to be sent to you separately"

The files required for the problem work unit (failure to decompress):

queue.dat 7 KB

client.cfg 1 KB (contains my personal folding configuration)

FAHlog.txt 36 KB

unitinfo.txt 1KB (not required, but contains info)

Plus, a Folder named:

- Work 3.89 MB

The Work folder contains various files related to the particular problem Work Unit

I would have to send these four files, plus the contents of the Work folder to you by email attachment.

These seven files and the "Work" folder should be placed in a directory

named "Folding@Home"

The main executable, FAH502-Console, runs in a DOS-like window and shows the progress, which is also recorded in FAHlog.txt .

The decompression failure occurs within a few seconds, up to a minute or so.

If anyone succeeds in decompressing and running the executable, for a few minutes. I would make the details known to the Stanford Folding Support Forum:

http://forum.folding-community.org/homepage.php

in the Windows 9X/ME section.

If you have any questions, please PM or email me <dnamechanic@gmail.com>

Thanks,

dnamechanic

---------------------------------------------------------------------------------------

LLXX, if you think this would be appropriate, I could post it in a separate thread,

with a link back to this thread.

I am open to other suggestions, maybe another forum?

[RainyShadow]

Have you tried running Folding@Home as a shell, without other programs running. If it still fails, probably better system won't make a difference. If it manages to decompress at least one unit this way, this would confirm that resource limitation (and heavy leaks) cause the program fail.

[eidenk]

I am open to other suggestions, maybe another forum?

This problem is a bug in either the folding home application per se or the decompression library they use IMO. And it can only be fixed by either the devs of folding home or the devs of the decompression library, depending on where the bug exactly is.

It is those people you should address IMO.

[dnamechanic]

This problem is a bug in either the folding home application per se or the decompression library they use IMO. And it can only be fixed by either the devs of folding home or the devs of the decompression library, depending on where the bug exactly is.

Thanks eidenk, for your thoughts.

I think I understand your point-of-view.

Yes, likely the F@H application or the decompression (BZIP2) library has a problem.

But, the problem only manifests itself to Win98 contributors.

Although, Win98 systems can process most of the Work Units issued by F@H.

I have observed that Win98-SE handles all such Work Units up to a compressed file size just below 3 MB. The Work Units that cause the V3Decompress error have a file size of approximately 3.5 MB.

> It is those people you should address IMO.

The F@H tech guys know about the problem.

It has been discussed here:

http://forum.folding-community.org/viewtop...90f509f4fb457e2

and several other threads at the F@H forums.

The bottom line is that they (F@H techs) have bigger problems to deal with.

The current F@H contributions for users of Win98 users are less than 5% of the total, and probably declining.

Also, most of Win98 users are using slower hardware (no offense ).

Seems as if many in this forum have fast machines.

I personally have Win 98-SE Folding on a couple of fast machines.

-dnamechanic

[dnamechanic]

Have you tried running Folding@Home as a shell, without other programs running. If it still fails, probably better system won't make a difference. If it manages to decompress at least one unit this way, this would confirm that resource limitation (and heavy leaks) cause the program fail.

No, RainyShadow, I had not tried. I never thought of it

So I proceeded to try it, running as a shell= statement in system.ini replacing:

[boot]

shell=Explorer.exe

with:

[boot]

shell=C:\FAH-502.exe

- All required files and folders placed in the root directory of C:\

- Disabled all start-up items, don't think they would be processed anyhow.

- Disabled all other unnecessary [boot] items in system ini.

It was an interesting scene, Windows 98 without the desktop.

Anyhow, F@H ran normally on a usual Work Unit.

Then, the V3decompress test files were loaded.

Sadly, it ended as usual:

------------------------------------------------------------------

[08:55:35] Folding@Home Gromacs Core

[08:55:35] Version 1.90 (March 8, 2006)

[08:55:35]

[08:55:35] Preparing to commence simulation

[08:55:35] - Looking at optimizations...

[08:55:35] - Created dyn

[08:55:35] - Files status OK

[08:56:51] Couldn't v3Decompress

[08:56:51] - Fatal: Could not decompress work unit data.

[08:56:51] Error: Could not open work file

[08:56:51]

[08:56:51] Folding@home Core Shutdown: FILE_IO_ERROR

---------------------------------------------------------------------

It was worth a try.

Thanks, RainyShadow, for the suggestion

-dnamechanic