Errors in CDs created by nLite?

[Multibooter]

I have created with Direct Burn+verify under nLite v1.4.9.1 a WinXP Home Edition SP3 installation CD which contains the AHCI driver. The CD worked fine installing WinXP SP3 on an Asus Eee 1000HE netbook. WinXP now runs fine on the 1000HE under both AHCI and IDE modes.

After installing WinXP I wanted to archive this CD by creating an ISO image of it with UltraISO v9.3.3.2685 under Win98SE, which is my main operating system on my main computer. At 100% complete, the iso-creation failed with the error msg: "CD/DVD Read Error!". I therefore repeated the creation and burning of the slip-streamed CD with a different CD-burner, starting with a freshly copied directory containing the original and unmodified WinXP installation CD. Again I got this "CD/DVD Read Error!" when I tried to make an ISO image of the 2nd slip-streamed CD.

Older UltraISO v7.6.5.1225 under WinXP, however, created an Ok iso file from the slip-streamed CDs, without this error msg. UltraISO v9.3.3.2685 has worked very reliably for me under Win98SE. I strongly suspect that nLite does NOT burn CDs correctly.

I then repeated the creation of this slip-streamed CD, and selected in nLite "Create Image" instead of "Direct Burn". I subsequently burnt a CD with Nero from this .iso image, and Nero produced a good CD, of which UltraISO could in turn make a good .iso image. So Nero burnt a good CD, but not nLite.

During that fiddling around I had actually created 4 slip-streamed CDs, which were supposedly identical. But when I compared the 6407 files on each CD with Beyond Compare/Hex Viewer, between 8 to 19 files on each CD were different, mainly just a few bytes, in IN_ and DL_ files. These were not bad burns because the files on the 4 CDs were identical to the files in the corresponding 4 working directories on the HDD used by nLite.

In computing, identical input should produce identical repeatable output, or a red flag comes up. WinXP installed fine with the 1st CD, but I have 3 more CDs, all different. Which one is a good one, with no surprises down the road, or are they all bad? Which one shall I archive? Why do some files in the working directories differ, given identical input?

[dencorso]

IN_ and DL_ files are disguised .CABs. I suggest you extract them and compare the extracted outputs. What you're seeing may be an artifact of the compressions. In any case, the IN_ files, when extracted, will yield INI files, that are ASCII text, so you should be able to find out exactly what differences they do have, if any. The DL_ files should give rise to DLLs, which are binary PEs, but Beyond Compare in hex mode with no alignment should offer good insights for them, too.

[Multibooter]

IN_ and DL_ files are disguised .CABs. I suggest you extract them and compare the extracted outputs. What you're seeing may be an artifact of the compressions.

Thanks dencorso. You solved the puzzle of the different files. When I extracted 6 of the different .CA_, .IN_, .SY and .TX_ files on 2 CDs, the resulting extracted files were identical.

[johnhc]

Multibooter, I have never had any trouble with CDs/DVDs burned with nLite. I have had trouble with an ISO (>2.6 GB) created by the default ISO builder but the optional mkisofs corrected this problem. I suspect you have a write speed problem. Try writing with a slower speed. Enjoy, John.

[Multibooter]

I suspect you have a write speed problem. Try writing with a slower speed.

Hi John,

Thanks for your thoughts, but the CDs were burned under nLite with the lowest burning speed (8x) and with Verify selected, and I strongly doubt that it is a writing speed problem.

Since the slip-streamed CDs displayed this "CD/DVD Read Error!" under Win98, but not under WinXP, I would speculate that the burning module of nLite burns a CD which is not quite compatible with Win98. UltraISO v9.3.3.2685 under Win98 could not create an ISO image of the CD produced by nLite [->Tools ->Make CD/DVD Image], but ->File ->Open CD/DVD worked Ok under Win98, without an error msg.

Since UltraISO under Win98 displays the error msg "CD/DVD Read Error!" when 100% of the nLite CD has been read, it might be something in the trailer which UltraISO cannot swallow under Win98, maybe it has something do with a slightly different treatment of file names or codepages under Win98.

I have never had any trouble with CDs/DVDs burned with nLite.

Have you tried to create under Win98 an ISO image of an nLite CD? I would venture to say that the output of the burning module of nLite has not been fully tested under Win98. Does nLite actually run under Win98SE if .NET Framework 2.0 is installed?

In any case, if it is confirmed, this is a really minor bug, and of interest probably only to the few remaining users of Win98. I was mainly interested in finding the reason for the different files, and dencorso has found the right explanation. The burning problem only led me to the puzzle of the different files.

Edited October 20, 2009 by Multibooter

[johnhc]

Multibooter, you make a good point. I have never even used Win98. You are probably correct about the two OSs conflicting but I would still try the lowest possible (is 8x really the slowest) and possibly be concerned about the interchange problem between the two optical drives. Have you ever run a cleaning disk on the drives? Perhaps the bottom line is simply that you have a workable solution, so stick with it. BTW, the nLite web site does not list Win98 as a supported OS, so you are most likely correct about the Win98 testing. Enjoy, John.

[Multibooter]

I have never even used Win98.

I use Win98 95% of my time, and WinXP maybe 5%. The main advantage of Win98 is its security from malware and government and industry spyware. I have been treading in dark waters, with a dedicated emule computer downloading under Win98 24 hrs/day, for the past 5 years. I only run on-demand scans, mainly of new downloads, encountering about 300-500 viruses, trojans etc per month. The last infection of my system was in January 2004, the last system-wide scan maybe a year ago.

msfn.org has the best Win98 forum in the internet at http://www.msfn.org/board/index.php?showforum=8 Here a "little" challenge: How about installing and creating a functioning Win98 on your current physical machine (i.e. not in a virtual machine)

Edited October 20, 2009 by Multibooter

[dencorso]

In computing, identical input should produce identical repeatable output, or a red flag comes up.

IN_ and DL_ files are disguised .CABs. I suggest you extract them and compare the extracted outputs. What you're seeing may be an artifact of the compressions.

Thanks dencorso. You solved the puzzle of the different files. When I extracted 6 of the different .CA_, .IN_, .SY and .TX_ files on 2 CDs, the resulting extracted files were identical.

While I'm really glad to have been able to help you, Multibooter, I fully agree with your words in the first quotation:

I know for a fact that non-lossy file compressors often create different files from the same input, which, when expanded, yield correctly the original input. I had observed this before both with cabarc and with pkzip, and your case is yet one more example of it. What I don't know is why. I guess random number generation may be involved in the earlier steps of LZW (and related algorithms) compression. But I never found the time to study them close enough to find out. So, for now at least, I remain baffled by it, although I'm sure there must be a simple enough explanation for it.

[Multibooter]

non-lossy file compressors often create different files from the same input, which, when expanded, yield correctly the original input. I had observed this before both with cabarc and with pkzip, and your case is yet one more example of it. What I don't know is why. I guess random number generation may be involved in the earlier steps of LZW (and related algorithms) compression. But I never found the time to study them close enough to find out. So, for now at least, I remain baffled by it, although I'm sure there must be a simple enough explanation for it.

I have recently come across another old program which had also raised a red flag with me, TeleDisk (see http://www.msfn.org/board/2-t136856.html posting #15, TeleDisk can create images of DOS and CP/M floppy disks under Win98 [TeleDisk v2.23 also works Ok under WinXP if you select as Diskette Controller Interface "BIOS" instead of "Direct"]): When I repeated with TeleDisk to make a disk image of the same floppy, the resulting disk image differed a little from the previous disk image, even the file size of the created image file differed. Making 3 images of the same floppy disk resulted in 3 different image files, but the floppies re-created from these differing image files were identical.

The floppy disk images usually just differed by a few bytes in the header, the data area looked identical. At the time I thought that they might have different encoded time stamps in the header and I did not pursue the issue further because the reputable hpmuseum site was using TeleDisk to archive their old CP/M floppies, e.g. http://hpmuseum.net/display_item.php?sw=243 with a downloadable file in .td0 (=TeleDisk) format.

Your guess about non-lossy file compressors may be the correct explanation of the different image files produced by TeleDisk, because TeleDisk can optionally create compressed floppy disk images.

cabarc, pkzip, TeleDisk and nLite seem to have something in common: identical input produces different output. A subsequent step, however, can produce again identical repeatable output. In the case of nLite, two seemingly different slip-streamed CDs (created with identical input) should create two identical Windows installations ... or maybe not?

It is probably easier to identify errors when identical input creates identical output. If identical input does not create identical output, eventually you get a pile of bugs, like Windows. And with every major new release, with new features, the number of bugs increases exponentially.

Edited October 20, 2009 by Multibooter

[johnhc]

Identical input to nLite should indeed produce identical installed Windows systems on identical hardware. I have used several different ISO creators (nLite default, ImgBrn and mkisofs) and all the ISOs are a little different in size. Here is a simple experiment on the compression question: take an .XX_ file from your original CD and extract it, then create a new .XX_ file using makecab (in Command Prompt). I think you will find that the created file is larger than the original but the two extracted files will be identical (I use the hash and size). Interesting invitation on Win98, but I am immersed in XP x64 and will stay here for as long as possible. Enjoy, John.

Edited October 20, 2009 by johnhc

[jaclaz]

It is possible that the original tool Microsoft used/uses is slightly different, but same app should create n times the same compressed file, unless an "external factor" like a "seed", a "salt" or whatever is part of the compression algorithm.

Something unrelated, but not much:

http://www.msfn.org/board/w98-slip-genuine...-9x-t81068.html

(about Diamond compression)

jaclaz

[Multibooter]

Here is a simple experiment on the compression question: take an .XX_ file from your original CD and extract it, then create a new .XX_ file using makecab (in Command Prompt).

I am using WinRAR to create archives, I have no experience with makecab. When I repeat the archive creation selecting a folder of files, WinRAR produces 2 identical .rar files. When I repeat the archive creation selecting a folder plus files, WinRAR produces 2 different .rar files, possibly because Windows changes the file access date when I select a file for inclusion in the .rar archive.

I think you will find that the created file is larger than the original but the two extracted files will be identical

I have been amazed to see that when I burn with Nero a CD from an .iso file created by UltraISO, and then make again an iso image with UltraISO of the CD just burnt by Nero, the resulting iso file will be larger than the original iso file, but the files in both iso images are identical.

Interesting invitation on Win98, but I am emerged in XP x64 and will stay here for as long as possible.

Pretty soon WinXP will be a legacy operating system, just like Win98. And legacy operating systems have something in common: It takes a lot of effort and special tools to get them to work on new hardware.

The installation of WinXP on an Asus Netbook, which has no floppy drive A:, has shown me that installing WinXP on new hardware starts to get as difficult as installing Win98 on new hardware. Without a tool like nLite I would not have been able to install WinXP on the Asus 1000HE netbook with an AHCI BIOS setting.

In 2-3 years from now I expect that it will be just as difficult to find WinXP-compatible hardware, WinXP drivers and WinXP-compatible software, as it now for Win98. If you plan to stay with WinXP, you might look at the Win98 forum http://www.msfn.org/board/index.php?showforum=8 to get a glimpse of the problems the few remaining Win98 users are currently facing.

[Multibooter]

Something unrelated, but not much:

http://www.msfn.org/board/w98-slip-genuine...-9x-t81068.html

Hi jaclaz, very interesting to see that this Win98 slip-streaming project died about 6 months after Microsoft stopped supporting Windows 98. Edited October 20, 2009 by Multibooter

[dencorso]

Here's a quite interesting site on compression I've found yesterday: Dr Ross's Compression Crypt. It's quite worthy of reading at lenght and musing about for a long time. But, especially noteworthy for owr current diacussion is this comment inside the source code of old_lzrw1-a.c:

4. Warning: This code is non-deterministic insofar as it may yield

different compressed representations of the same file on different

runs. (However, it will always decompress correctly to the original).

There I learnt that algorithms that yeild consistently the same compressed output for a given input are said to be deterministic, and the meaning of non-deterministic is clear from the above quote. Now, digging a little deeper, one may find out that PKZIP, WinZip, WinRAR and 7-Zip, all use multiple compress algorithms, during normal real-world use, when compressing multiple files (and not all these algorithms are deterministic), selecting automagically the best one for each given file in the input, so that the result may be non-deterministic, when at least one file was compressed with a non-deteministic algorithm, or can be deterministic, when only deterministic algorithms have been used. The Wikipedia pages for WinZip, PKZIP, DEFLATE (one of the Zipping algorithms) and 7-Zip are also very enlightening to read.