W98_Slip: genuine slipstreaming for windows 9x

[glaurung]

W98_Slip: Slipstreaming Windows 98 SE.

About:

W98_Slip is a set of batch and script files used to slipstream updates into Windows 9x. Currently it has only been tested with Windows 98se, but with minor modifications it should work with Windows ME, Windows 98 first edition, and (with more extensive modification) maybe even Windows 95.

[eta: Its main limitation is that it can only slipstream updated files. New files could be slipstreamed, but it would take editing by hand of the makecab script files.]

Credits:

W98_Slip is written by Glaurung. This project was inspired by TommyP's HFSlip project.

Contact info:

For information and support, visit the W98_Slip thread in the MSFN Windows 98se unofficial service pack Forum.

Version history:

This is version 0.1 (first release).

Copyright and credits:

These files are open source, released under the GPL. You may use them for any use, personal, non-profit, or commercial. But you do so with the understanding that, 1) these files are experimental, unwarranteed, and you use them completely at your own risk, and 2) using these files may invalidate your licence with Microsoft, and any license issues caused by using these files are totally your responsibility.

Currently, these files are entirely my own work; I welcome contributions by others. You may redistribute these files, but if you do so, please give me credit, and leave in the contact information so users know where to go for information and updates.

Warnings:

These have been tested successfully with my OEM copy of Windows 98 SE. I have used them to do a maximum install from both DOS and within Windows with no error messages whatsoever. However, I don't own a retail copy or a bulk licence copy to test on. These files may or may not work with your copy of the OS.

There is no guarantee, and no warrantee. There is no support. This project is something I did in my spare time. I'd like to help you if you have problems, but I may not be able to reply right away or at all.

Another warning:

My batch file writing skills are pretty basic. Currently there is little error checking or sanity control. If you follow the directions, it should work. If you don't, it won't work.

w98_slip: Instructions for use.

0. You will need: 7zip or winzip, to extract updated files from hotfixes or from the unofficial service pack. Makecab (Microsoft cabinet SDK), for obvious reasons. Minitrue, used by W98_Slip to turn file lists into makecab scripts.

1. Unzip the files to a new folder on a disk with 1gb or more of free space. I've always put them in a top-level folder with a short file name. They might work for you if you put them in "My documents/my very long folder name with lots of spaces," and then again they might not.

2. Run w98_slip.bat. It will create some subfolders and display a short version of these directions.

3. Copy the win98 folder from your Windows 98 SE CD to the "source" folder.

3a. W98_slip expects some files to exist that are not required for setup to work properly. If the copy of the setup files that you have handy doesn't include setup.txt or setup*.wav, then when W98_slip tries to compress the updated source, makecab will fail with an error. If you can't find your original win98 CD, just run "makecab /f pass2.ddf" to see the missing files, then create some zero-byte dummy files with the same names.

4. Put the new versions of the files you want to update in the "updates" folder.

4a. If you want Windows Update to see the hotfixes you're slipstreaming as already applied, you need to edit an existing .inf file in the source to add the registry entries that tell windows update that you're already patched on that issue. Sorry I can't be more specific but the exact entries to make will depend on the exact files you are updating.

4b. Two restrictions on updated files: a) the files in the "updates" folder cannot be compressed (extract them first), and B) 98slip will only update files that already exist in the original (ie, w98_slip won't slipstream the q891711 update because it uses new files not in the original). You can add files not found in the original, but you'll have to edit the scripts by hand, and edit inf files in the source to take account of the new file.

5. Put the following programs in the "tools" folder: makecab.exe (downloadable as part of the Microsoft Cabinet SDK), and mtr.exe (part of minitrue, downloadable from http://www.idiotsdelight.net/minitrue/). Also, if you have used 98lite (pay version) to remove the DOS command line utilities from windows\command, reinstall them.

6. Run w98_slip.bat again. The program will pause at various points, both to let you see if any errors popped up and to allow you to cancel out if need be, so it's not possible to run it totally unattended.

7. The slipstreamed install files will be in slipped\win98.

8. If you run w98_slip.bat again, it will ask you if you want to reuse your previous decompressed source. This is mainly a time saving measure, since decompression takes several minutes. It will also ask you if you want to reuse your previous makecab scripts. This is in case you modified the scripts for whatever reason. Finally, it will ask if you want to recreate the entire slipstream or just the precopy.cab files. Again, this is a timesaving measure for people who mainly want to modify the .inf files.

Things to note:

1. The install files will not be the same sizes as the originals, and there will be slightly fewer of them. This is because w98_slip is designed to generate the same number of cab files regardless of the size of your updated files.

2. In your slipstreamed install, Sfc.exe may not work properly (I haven't tested it so am unsure), as it contains hardcoded references to specific .cab file names. Fixing this would require hacking the binary file default.sfc.

What the files do:

pass1.ddf and pass2.ddf are makecab directive file stubs (everything makecab needs except the list of files to compress). The file lists for makecab are generated by w98_slip on the fly using minitrue.

mt*.txt are minitrue scripts. mt.txt, mt1.txt, and mt2.txt handle the conversion of the three layout.inf files to makecab directive files. mt_fl.txt and mt_pc.txt handle the conversion of the list of files contained in the cabs into makecab directive files.

mt_inf.txt handles the problem that driver19 and driver20.cab don't exist in the fileset w98_slip produces, but are referenced in a couple of inf files. I made it external to help simplify the process of adapting w98_slip to work with other versions of the OS.

w98_slip.zip

Edited August 20, 2006 by glaurung

[glaurung]

Background discussion:

Leaving out all the false starts and frustrating attempts to wrap my head around what passes for makecab documentation, this is a rundown of some of what I've learned doing this project, which may be of use to you if you want to slipstream another version of windows or use it as a springboard to create a more ambitious Windows 9x slipstreaming tool.

Using extract /d to list the contents of precopy2 reveals that the first file is carried over from precopy1, which is, mysteriously, located on "disk 2." Doing the same with base5 reveals that the first file is carried over from base4 on "disk 3." (we can safely assume that "disk1" consisted of the uncabbed setup files like setup.exe, *.bin and the like).

Looking at the sizes of the cabinet files, most of them are all the same size (1760 k). However, some are smaller; win98_21 is less than 6 k. On the other hand, driver20 plus win98_21 total almost exactly 1760k. More arithmetic shows the other odd-sized cabs also add up to almost exactly 1760k if you group them in numerical order.

So it seems that Microsoft used ".Set MaxDiskSize=1802240" in their makecab ddf file. Makecab then fit as much onto each "disk" as it could, resulting in the odd sized cabs. Using maxdisksize doesn't make much sense, because windows 98 was never released on floppy disks. I guess they adapted the ddf file they used for windows95, and simply never bothered to switch from a MaxDiskSize setting to a MaxCabinetSize setting.

As to the size they chose: since large cab files take longer to process, Microsoft had to keep the cabs to a relatively small size so that Windows would install in a reasonable amount of time on a 486. On the other hand, I assume they wanted to keep the cabinets larger than the maximum capacity DMF floppy disk to keep people from making copies of the OS on floppy disk (this was in the days before CD-R, remember). (ETA: LLXX tells me that 1760k is the size of a floppy format that Microsoft developed but abandoned due to compatibility problems. Probably makecab supports this format with an undocumented abbreviation, just like it supports the abbreviations 1.44M and CDROM).

Opening the source cabs with 7zip shows how the files are grouped in folders (7zip calls them "blocks"). Extracting the files from a single block and re-cabbing them yields cabs of about 1 megabyte. Exactly one megabyte yields folders a little larger than the source cabs; a little experimentation shows that Microsoft used: ".Set FolderSizeThreshold=1000000." This is actually pretty close to optimal: setting the value larger doesn't result in any significant savings.

(NB: what passes for makecab's "documentation" has an example where "1m" is used to stand for 1000000. This is wrong: using the abbreviation generates an error).

Using extract /a /d to process the entire cabinet chain reveals that precopy* is on one cabinet chain, while catalog3 through win98_74 is on a second cabinet chain. Chl99.cab, Win98_ol.cab, and mini.cab are separate.

So it seems that Microsoft created the source cabinets in two passes. First, they created a complete set of source cabs, from precopy1 to win98_74, using dummy layout.inf files. Then they took the layout file generated in that pass, sliced it into three, and ran a second pass to generate precopy*.cab with the real layout.inf files.

A closer look at the file dates shows that the following files share the same file date as layout*.inf, which is different from the file date for all the other files. Presumably these differently dated files are linked somehow, since they were all updated together: copy*.inf, del*.inf, ren.inf, and default.sfc.

With that as background, there are several hurdles to slipstreaming:

1. Windows setup copies the cabs to a temporary directory on the hard disk before it extracts their contents (remember this was in the age of 1x cd-rom drives). Precopy1 and Precopy2 are hard coded (you can find the strings in w98setup.bin). Cabs from catalog3 through driver20 are soft-coded in copy*.inf, predrv.inf, and setupc.inf, but don't appear to be hard coded. Cabs from win98_21 on are soft coded in setupc.inf.

So our makecab script has to ensure that two and only two precopy cabs are made, and that we will always have the same number of cabs regardless of the size of the updated source files. W98_slip does this by setting maxdisksize to CDROM and using new cabinet directives at the same points that new cabs began in the original. It then edits the three soft coded inf files because the resulting fileset was smaller due to some large files spanning several cabs in the original.

2. The layout.infs don't list the [sourcediskfiles] in the order in which they appear in the cabs. I don't know why this is, but it means that using the default layout.inf generated by makecab (which lists the files in the order in which they appear in the cabs) results in "cannot find file" errors during the file copy phase of setup. Again, I don't know why this is, but my best guess: the layout*.inf files are required to be in synch with the other files with the same timestamp (copy*.inf in particular).

So our makecab script has to be a relational script, which orders its layout.inf file differently from the order of files in the cabs, using ".Set GenerateInf" directives.

3. Relational .ddfs require all files you're compressing to be unique: you can't put command.com into precopy1.cab and again into base4.cab. This is exactly what Microsoft did (there are 3 files duplicated between precopy and base, and another file duplicated between the list of uncompressed setup files and the cabs). So w98_slip uses the workaround of putting the duplicate files in under different names during pass one, then editing the generated layout file to put the real names back, and putting the duplicates in under their own names during pass 2.

4. Things that may be a hurdle: some of the files in layout.inf have a CRC appended to them. For windows 98se, the CRC appears to be optional. That may not be so for ME. I know it's possible to have makecab append the crc to specified files in layout.inf using the "source_file /x=y" switch, but the makecab "documentation" is even less useful in explaining the /x=y switch than usual.

W98_Slip uses minitrue (an alternative to sed) to automatically generate the bulk of the makecab directive file on the fly. The advantage of this (besides tiny download size) is that it should be relatively easy to modify to work on other versions of the OS.

Edited August 20, 2006 by glaurung

[LLXX]

Excellent write-up on the details of the Windows 9x setup distro package

FYI 1,802,240 (1.72M) is actually the capacity of an obscure floppy format that Micro$oft created, but it was found that some floppy drives could not read it, so it was abandoned. I have attempted to recreate this format, and my floppy drives can read and format it fine, but other drives may have problems with 84 tracks.

Volume XTDTOOLS created 04-07-2004 4:16p
Volume Serial Number is CF64-C47A

	1,802,240 bytes total disk space
	1,802,240 bytes available on disk

	   32,768 bytes in each allocation unit
		   55 total allocation units on disk
		   55 available allocation units on disk

The exact parameters are 84 tracks and 21 sectors/track, 32k cluster size, one root directory sector, two fat sectors.

[jaclaz]

Yes,

I may add that besides hardware problems, this floppy format, as well as ANY floppy format with more than 80 tracks CANNOT be made under NT/2K/XP/2003 due to the HAL and drivers.

(and besides you cannot even properly format a disk under XP):

http://www.denispetrov.com/format144/

See for reference my post here:

http://www.serverelements.com/phpBB2/viewtopic.php?t=64

and Winimage's author Gilles Vollant:

http://www.winimage.com/wimushlp/wini1a1y.htm

And some links to (dos/win9x) programs:

Nformat:

http://toastytech.com/files/nformat.html

fdformat:

ftp://ftp.simtel.net/pub/simtelnet/msdos/...til/fdfrm18.zip

SmartFormat:

http://members.tripod.com/ieeecrce/innovations99/tformat/

and to dcopy 2.8 (the one I use):

http://users.pandora.be/jbosman/applications.html

There is also the 2M program that gives even better capacity (but it needs a special driver):

http://esca.atomki.hu/paradise/sac/utildisk.html

ftp://ftp.sac.sk/pub/sac/utildisk/2m30.zip

ftp://ftp.sac.sk/pub/sac/utildisk/2mgui19.zip

But I never heard about the 1,802,240 size, the math does not fit:

2x84x21x512= 1,806,336

Can you post more details?

jaclaz

[LLXX]

Indeed, XP 'formatter' is a joke. All the formatting I do is with Fint13 from DOS.

A special driver is not needed to support >18 sectors/track, just set the floppy in BIOS as 2.88Mb.

1802240 is the formatted capacity, just like a "1.44Mb" floppy actually has 1457664 usable bytes but 1474560 actual bytes in sectors.

1802240 bytes = 3520 sectors of usable space

+ 512 bytes = 1 sector for root directory

+ 1024 bytes = 2 sector for 2 FATs

+ 2048 bytes = 4 reserved sectors

+ 512 bytes = 1 sector for superblock

= 1806336 bytes = 3528 sectors total

[glaurung]

Excellent write-up on the details of the Windows 9x setup distro package

FYI 1,802,240 (1.72M) is actually the capacity of an obscure floppy format that Micro$oft created, but it was found that some floppy drives could not read it, so it was abandoned. I have attempted to recreate this format, and my floppy drives can read and format it fine, but other drives may have problems with 84 tracks.

Thanks for the thumbs-up.

Well, that explains the size of the cabs; I suspect that this format, along with the DMF format, have abbreviations in makecab (like the the other floppy sizes) which aren't mentioned in the "documentation." Then again, there's a lot that's not mentioned, or mentioned only in the most obscure manner, in the makecab documentation.

[glaurung]

Edited my original post to add instruction #3a, about how w98_slip expects some unnecessary files, such as setup.txt and setup*wav, to exist in the source.

[wizardofwindows]

outstanding piece of work thus far.be watching future developments

[LLXX]

Excellent write-up on the details of the Windows 9x setup distro package

FYI 1,802,240 (1.72M) is actually the capacity of an obscure floppy format that Micro$oft created, but it was found that some floppy drives could not read it, so it was abandoned. I have attempted to recreate this format, and my floppy drives can read and format it fine, but other drives may have problems with 84 tracks.

Thanks for the thumbs-up.

Well, that explains the size of the cabs; I suspect that this format, along with the DMF format, have abbreviations in makecab (like the the other floppy sizes) which aren't mentioned in the "documentation." Then again, there's a lot that's not mentioned, or mentioned only in the most obscure manner, in the makecab documentation.

The abbreviation is DMF168.

Also, I have found what appears to be the 'full' documentation for Diamond , as well as Diamond.exe, from an old Java SDK. Diamond must be the original program used by M$; unfortunately, the documentation and program are from 1994 so it probably does not have newer features, but supports a "Quantum" CompressionType. If you want I can upload it somewhere.

Quite a coincidence that MakeCAB and Diamond are both exactly 7 letters long.

Here are some of the interesting strings from Diamond.exe:

681984000 65535 CDROM DMF168 1716224 2048 16 1.68M 1457664 1250304 192 1.25M 1213952 512
224 1.2M 730112 720K 362496 1024 112 360K Set Option New InfWriteDisk InfWriteCabinet InfWrite InfEnd
InfBegin Dump Delete Define UniqueFiles SourceDir setup.rpt RptFileName ReservePerFolderSize
ReservePerDataBlockSize ReservePerCabinetSize 20 MaxErrors MaxDiskSize MaxDiskFileCount MaxCabinetSize
DCF InfSectionOrder InfHeader6 InfHeader5 %1** Diamond Version: %3 InfHeader4 InfHeader3 InfHeader2
InfHeader1 InfHeader InfFooter4 InfFooter3 InfFooter2 InfFooter1 CompressionMemory CompressionLevel
CompressionType Quantum Compression Level not in range (%1..%2): %3
Compression Memory not in range (%1..%2): %3
Microsoft (R) Diamond Disk Layout Tool - Version %1 Copyright (c) Microsoft Corp 1993-1994.
(32) 1.00.0520 (12/16/94) Compression will be slower. A few quarts low on memory. Compression will be
much slower. Critically short of memory.

From the documentation:

Quantum is a new compression technology that Microsoft obtained an unrestricted license to in early May, 1994. It achieves compressed file sizes 10-15% smaller than MSZIP, and Quantum will be the preferred compressor (possibly the only one) supported by Diamond. In order to achieve these impressive results, Quantum can require a fair bit of memory (up to 12Mb) at compress time, and even at decompress time (configurable from 1K to 2Mb), and Quantum gets its best results on large data streams. For this reason, cabinet files and Quantum are a great fit, because cabinet files with large folders ensure that Quantum is always compressing big blocks of data. The decompression memory requirements for Quantum is tunable in the Diamond directive file (see the CompressionMemory variable).

CompressionLevel=1 | 2 | ... | 7

Selects the Quantum compression level.

Default: .Set CompressionLevel=2 ; Default is level 2

The lowest setting is 1, and Quantum yields the least compression and runs the most quickly (for a given value of CompressionMemory). The highest setting is 7, and Quantum yields the highest compression, but runs for quite a long time.

See the table in the CompressionType variable definition for a comparison of various settings.

This variable is ignored if Compress is OFF or if CompressionType is not set to Quantum.

Examples:

.Set CompressionLevel=7 ; Set maximum compression level

CompressionMemory=10 | 11 | ... | 21 | 1024 | ... | 2097152

Selects the amount of memory required by Quantum at decompress time.

Default: .Set CompressionMemory=18 ; Default is 18 (256K)

The value can be specified as either a power of two exponent, or as a byte count. If a number in the range 10 to 21 is specified, Diamond raises 2 to this power, yielding memory sizes in the range 1024 (1Kb) to 2,097,152 (2Mb) bytes. If the number is in the range 1024 to 2097152, it is treated as a byte count, and is rounded up to a power of 2 (if it isn’t already).

See the table in the CompressionType variable definition for a comparison of various settings.

This variable is ignored if Compress is OFF or if CompressionType is not set to Quantum.

Examples:

.Set CompressionMemory=21 ; Set maximum compression memory

[oscardog]

LLXX If you could upload diamond somewhere it would be greatly appreciated, I was puzzled why i could not create cabs in parity with the setup ones

glaurung thanks very much for the slip.bat and the background info very useful

[Petr]

I have located "Microsoft ® Diamond Cabinet Builder - Version (32) 1.00.0540 (02/01/96)" in IEAK 3.2

It is here: diamond.zip

Edit:

And what is interesting, there is three exactly identical files, just the name is different:

diamond.exe

diant.exe

diantz.exe

DIANTZ.EXE is part of Windows 2000 / XP / 2003 and it is almost identical to MAKECAB.EXE, even the help (in XP) is identical:

C:\WINDOWS\system32>diantz
Microsoft (R) Cabinet Maker - Version 5.1.2600.2180
Copyright (c) Microsoft Corporation. All rights reserved..

MAKECAB [/V[n]] [/D var=value ...] [/L dir] source [destination]
MAKECAB [/V[n]] [/D var=value ...] /F directive_file [...]

Petr

Edited August 21, 2006 by Petr

[Acheron]

See also http://www.kyz.uklinux.net/cabextract.php for more information about Cab Compression history and extractors.

[glaurung]

Also, I have found what appears to be the 'full' documentation for Diamond , as well as Diamond.exe, from an old Java SDK. Diamond must be the original program used by M$; unfortunately, the documentation and program are from 1994 so it probably does not have newer features, but supports a "Quantum" CompressionType. If you want I can upload it somewhere.

I'd be interested in seeing the documentation. The exe, not so much.

Also, I seem to remember once seeing reference to a tool that would look at a cabinet file and automatically recreate the .ddf used to create it. If you found any sign of that in that old Java SDK, let me know.

[jaclaz]

Indeed, XP 'formatter' is a joke. All the formatting I do is with Fint13 from DOS.

A special driver is not needed to support >18 sectors/track, just set the floppy in BIOS as 2.88Mb.

1802240 is the formatted capacity, just like a "1.44Mb" floppy actually has 1457664 usable bytes but 1474560 actual bytes in sectors.

1802240 bytes = 3520 sectors of usable space

+ 512 bytes = 1 sector for root directory

+ 1024 bytes = 2 sector for 2 FATs

+ 2048 bytes = 4 reserved sectors

+ 512 bytes = 1 sector for superblock

= 1806336 bytes = 3528 sectors total

Thanks LLXX, do you also happen to know if it uses 1, 2 or 4 sector(s) cluster?

(the 1.68 DMF format differed from "normal" 1.68 as it had two sub-versions one DMF1024 with two sectors clusters and DMF2048 wit 4 sectors clusters)

jaclaz

[jimmsta]

DIANTZ.EXE is part of Windows 2000 / XP / 2003 and it is almost identical to MAKECAB.EXE, even the help (in XP) is identical:

C:\WINDOWS\system32>diantz
Microsoft (R) Cabinet Maker - Version 5.1.2600.2180
Copyright (c) Microsoft Corporation. All rights reserved..

MAKECAB [/V[n]] [/D var=value ...] [/L dir] source [destination]
MAKECAB [/V[n]] [/D var=value ...] /F directive_file [...]

Petr

Diantz may be using an earlier version of the CAB compression spec. "21xx" is found throughout diantz, while makecab has "22xx" (hexadecimal values in file). The files are nearly identical. There's a small section of data within the exe that is approx 16bytes, which is different between the two files. This may be the algorithm used. I have no idea though, as I'm not a real hacker/programmer...