137GB limit - ESDI_506.PDR and other limits

[MDGx]

check out this article by Fred Langa that talks about Drive Size Limits:

http://www.langa.com/newsletters/2006/2006-04-10.htm#4

Yet more misinformation...

That segues into software, which can have exactly the same kind of mathematic limitations. The old DOS FAT12 system, with a 12-bit addressing scheme, could only handle drives to 16.7MB in size; DOS3 brought this to 33.6MB; DOS4 to 134MB; Fat16 has a 2.15GB maximum; FAT32 a 32GB max; and so on.

I clearly remembering using DOS 4.01 with a 300Mb hard disk... the actual maximum limit is 2Gb.

You also can run into problems with non-OS software: The Win98 Format tool, for instance, maxes out at 64GB due to its own, separate internal limits.

Isn't it 128Gb?

If I recall correctly DOS 4.0 indeed came with FAT12, which used to support [officially] HD partitions up to 16 MB and unofficially supported HD partitions up to ~ 32 MB [?].

But not long after that, MS-DOS 4.01 came out with FAT16 [even if it still was advertised as FAT12], which used FDISK with support for FAT16 [not advertised as such]. That version of FDISK supported HD partitions up to ~ 128 MB. But FORMAT.COM from MS-DOS 3.3/4.0x only supported up to 16 MB partitions.

Please see this for details:

http://www.mdgx.com/secrets.htm#FAT32

FYI:

FAT32 [the newer FAT32X edition (1997) implemented in Windows 95C OSR 2.5] partitions can go up to 2 TB [Terabytes] if proper BIOS [or BIOS wrapper] support is provided [48-bit LBA (Logical Block Addressing)].

More stuff here, but not complete:

http://en.wikipedia.org/wiki/MS-DOS

HTH

[Petr]

But not long after that, MS-DOS 4.01 came out with FAT16 [even if it still was advertised as FAT12], which used FDISK with support for FAT16 [not advertised as such]. That version of FDISK supported HD partitions up to ~ 128 MB. But FORMAT.COM from MS-DOS 3.3/4.0x only supported up to 16 MB partitions.

I remember that I was using 33 MB partitions with MS-DOS 3.3.

Please check the following references:

http://support.microsoft.com/kb/69912

http://mirror.href.com/thestarman/asm/mbr/FDISK.htm

Petr

[DOS Forever]

OK, I've read this entire thread and have been doing some experiments. Perhaps someone can point out what I'm doing wrong or right.

I used the following hardware:

ASUS P5RD1-V (AMI BIOS, ULI M1573 chipset, supports 48-bit addressing)

Seagate ST3160023A (160GB, ATA, UDMA5, supports 48-bit addressing)

I partitioned and formatted the drive with Seagate's software, CHS = 19457x255x63, FAT32. The drive is configured as 2 partitions (C: & D:), each 80 GB. I did NOT install Seagate's "Dynamic Drive Overlay" (DDO).

I then temporarily installed a 2nd hard drive (40 GB WD) and booted with a WinME startup disk. I created 8 directories on each partition and copied approximately 10GB of data from the 2nd drive into all directories of the second partition (and only on the 1st directory of C:). I used the /v to verify the copy operation. After each copy, I ran scandisk and had no errors. While filling the last directory, #8, I ran out of space (as expected).

I did a FC /b of all files, comparing the 1st and 7th directories - no differences found. I then deleted all files in the 8th directory, and again did a FC on the 1st and 7th directories - no differences found.

So it appears that my data is fine. There does not seem to be any sort of "wrap-around" occurring. DIR is reporting the correct drive and directory sizes. After deleting all the files in the 8th directory, I regained the correct amount of space. Everything seems correct.

Is this because I was using WinME instead of Win98? (I did not have a Win98 startup disk handy) Is it because I'm running in DOS mode? Is it because I have partitioned the drive into two 80GB partitions? Are there some other tests I should run? I'd like to install Win98 on this machine/drive and use it, but I do have some concerns and value my data.

Comments appreciated!

......gary

[Petr]

The >137GB problem is identical for Win98, Win98SE and WinME.

Have you checked what driver is used? Is it really ESDI_506.PDR?

Then you can try 48BITLBA program to check the status of 48-bit LBA support of your system.

It is part of Mr. Loew 137GB patch package, direct download link is here: http://members.aol.com/rloew1/Programs/Patch137.zip

One thing is not very clear in your post - you wrote "I booted with a WinME startup disk" - have you booted really to Windows (GUI) or to command prompt only? In fact, command prompt (not DOS box inside the Windows) uses BIOS calls only and no problem appears.

The problem exhibits just and only if you use 32-bit disk access using ESDI_506.PDR driver.

Petr

[DOS Forever]

One thing is not very clear in your post - you wrote "I booted with a WinME startup disk" - have you booted really to Windows (GUI) or to command prompt only? In fact, command prompt (not DOS box inside the Windows) uses BIOS calls only and no problem appears.

The problem exhibits just and only if you use 32-bit disk access using ESDI_506.PDR driver.

Petr

Thanks for the quick response!

Well I think you just explained it. I never actually loaded windows, and was only running from the command prompt. So the problem only exists with the driver used by the windows GUI - not when running in its true DOS mode. Thanks for the clarification.

I also have an add-in card with a driver. However I'd rather not use this, as my BIOS already supports 48-bit addressing. It would be a shame to install it just to utilize its driver.

Has anyone developed a fix or work-around for this yet?

......gary

[eidenk]

Has anyone developed a fix or work-around for this yet?

Microsoft should have fixed that long ago. When they fixed it for Win2K in fact. They could be sued millions for that if the game was any fair IMO.

[LLXX]

Has anyone developed a fix or work-around for this yet?

I'm working on it... though I'm quite busy and haven't really gotten the uninterrupted time I need to disassemble, analyse, and completely rebuild the driver. Maybe in the summer... but I haven't forgotten about it yet

[CLASYS]

Can anyone compile a complete list of which hardware/software combinations are capable of supporting larger than 137 GB disks?

As I understand it, all the following is true:

1) Any one partition on the disk cannot be bigger than 137 GB, largely due to restrictions within Scandisk and Defrag. Any other problems per se? [Would there be alternative programs that don't have the problem, such as DISKKEEPER and/or some Symantec/Norton utility? I know that certain Symantec packages subsititute I believe Norton Disk Doctor for SCANDISK throughout Windows. And what of the DOS-based SCANDISK that runs on 98/98SE [not relevant on ME!].

2) It doesn't matter where on the disk these partitions are, just the size of any one of them; split the disk up any way you feel best.

3) DOS and safe/forced compatibility mode don't need anything fixed; they just work. A question: What about defraggers for DOS and large disks? And what of DOS chkdsk and scandisk [pick any relevant versions from MS-DOS, PC-DOS, FREE-DOS, DR-DOS, etc.].

4) 32-bit mode needs a 48-bit-aware driver:

a) Mr Lowe sells a patched ESDI_506.PDR version for most systems. [i am a registered user.]

b ) Intel application accelerator works for certain specific Intel chipsets [can anyone give the exact models?]

c) Perhaps something from VIA does this, or does it merely interfere with anything else? [Mr Loew intimates that perhaps it not only doesn't work, it prevents his item from functioning. He also disputes the 98SE hotfix to correct FDISK for larger disks actually completely works beyond some specific size that could be relevant here.]

d) I would appreciate something for my AMD chipset-based system [that is skirting the issue by using a 120 GB disk currently]. There are drivers from AMD, but I don't know if any of them deal with 48-bit.

e) Just about anything USB 2.0 or firewire external hard disk in a box just works in Windows, as long as there is an appropriate driver. However, for DOS, the driver situation is muddled; some drivers work fine for some hardware configurations, but it could be quite specific. Some drivers hate certain card/disk combinations, some hate certain cards period. Some drivers only support 127 GB and down, others work totally fine. In my own experience, the recommend DOS drivers are those associated with the fully live-updated version of Ghost2003, allegedly coming from IOMega, to Symantec. The firewire driver works amazingly well, on such disparate hardware as Sony VAIO laptops, to common garden variety VIA chipset firewire 400, to SIIG's 64-bit transfer PCI FireWire-800 card that in my AMD system causes noticeable throughput improvement in the 64-bit PCI slots, even from DOS! [using FW800 hardware throughout.] But cheap ALI chipset USB cards only work if the disk is less than 127 GB. The driver totally hates the VIA chipset built into Tyan 2495ANRS USB 2.0 regardless of disk size. And some drivers hate intervening USB hubs! All USB cards based on NEC chipsets seem to fare the best, including DOS large disks, etc.

f) Add-in cards such as the Promise TX2-Ultra 100 and 133 work fine supplying their latest drivers to 98SE. [Note to owners of the 100 version: Use the 133 card's driver; the Promise website is misleading on this point.] I use the Promise driver to access larger disks just fine in XP as well as 98SE. [Not a boot drive for 98SE though. XP in theory could do it it is claimed.] [Note: XP will not identify the 133 card at all, but it will misidentify the 100 card; beware of not using the driver!] BIOS support for DOS works fine [upgrade the ROM to the latest version on their website.] Anyone tried SIIG, Addonics or High-Point equivalents?

To my knowledge, the only known 98SE-friendly SATA II [300] card comes from Addonics, but there are some SATA I cards that are. Presumably certain machines have SATA support in the BIOS that might work with any of the above. In any case, these are on a case-by-case basis as to whether they support 98SE and hard disks of any size, much less greater than 137 GB.

A side note: Partition Magic 8.05 will refuse to make a FAT32 partition bigger than 196 GB. Can anyone explain this number? [FDISK from a 98SE startup disk will make a full 250 GB into one partition.l] Partitions larger than that number seem to give WinXP some problems as well [perhaps that's the explanation!?].

I apologize for any errors in the above. Can someone correct/elaborate/add on other hardware known to either work or not work, etc.

cjl

[Petr]

Can anyone compile a complete list of which hardware/software combinations are capable of supporting larger than 137 GB disks?

I think almost everything was written in this thread already.

1. AFAIK the problem is just with defrag and scandiskw applications. I should be noted that this problem occurs for ANY type of the disk, i.e. even for SCSI or USB disks.

2. If you place all partitions used by Windows 98 below 137 GB barrier you can use even bigger disk even if the Windows 98 driver does not support 48-bit LBA (e.g. in multi-boot configurations with XP or Linux)

3. Forced compatibility mode is almost unusable because of big slowdown and delays. DOS Scandisk is reported to work for bigger disks.

4. ATA (IDE) disks bigger than 137 GB requires 48-bit addressing to access the space beyond 137 GB.

a) Mr Lowe sell just patch for ESDI_506.PDR, ESDI_506.PDR is generic 32-bit driver for ATA HDDs.

B) Intel® Application Accelerator - Supported Chipsets - in other words it supports the following Intel southbridges: ICH - 82801AA, ICH-0 - 82801AB, ICH-2 - 82801BA, ICH-4 - 82801DB. ICH-3 - 82801CA was supported by version 2.2.2 only.

In addition to these chipsets it apparently supports also older chipsets, the INF file is hidden inside of the IAA23 package. The southbridges are: PIIX (82371FB), PIIX3 (82371SB), PIIX4 (82371AB), PIIX4E (82371EB). It means support of 430FX, 430HX, 430TX, 430VX, 440LX, 440ZX, 440EX, 440GX, 440ZX, 450NX. According to official statement Intel® Application Accelerator does not support the Intel® 440BX/LX/EX/GX/FX or 430xx chipsets but I have tested in on several computers with 430HX, 430TX and 440BX chipset with no problem so far.

It should be noted that Ahead recommends to uninstall IAA - very clever instruction if you have disk bigger than 137 GB!

c) Regarding VIA, ULi (ALi), SiS, nVidia drivers I have no personal experience.

d) Just connect bigger disk and use Mr. Loew utility.

e) I'm not sure what you are discussing, functionality in Windows 98 or in DOS? I just tried several USB2 controllers, NEC with 720101 chip seems to be OK but slow, VIA VT6202 is OK but a bit slow too, VIA VT6212 is the best, card with ULi/ALi chip M5273 did not work at all, both with Orangeware and ULi driver, both in Windows 98 and XP. My biggest USB disk is 120 GB so I cannot say how they work for >137 GB disks.

f) Yes, this Promise driver can be used even for Ultra66. Unfortunately BIOS in Ultra 66 does not support 48-bit LBA. As of other manufacturers, I know that Slicon Image (former CMD Technology) PCI-0649 does not support 48-bit LBA (neither BIOS nor Win9x driver) and newer PCI-0680 supports 48-bit LBA (both BIOS and driver). Apparantly current Highpoint controllers support 48-bit LBA and Windows 98.

It would be nice to have the table with all ATA/IDE controller chips showing the 48-bit LBA support in BIOS and 48-bit LBA support in Windows 9x driver.

Probably 48BITLBA.EXE - 48 Bit Support Test Program Version 4.6 from Mr. Loew could be used for testing?

Petr

[LLXX]

1. I've heard that the DOS scandisk has been tested on disks up to 512Gb with no problem.

2. Correct.

3. This is because they go through the BIOS, and most BIOSs from 2000+ support the Int13x interface that is compatible with 48-bit LBA. I have not found a DOS FAT32-capable defragger yet.

4. Patched ESDI_506.PDR version of Loew can actually be somewhat dangerous, he does not add any code to check if the drive supports 48-bit LBA, thus the driver may fail if used on a drive that does not support the extended commands. I consider this unacceptable, as I frequently swap drives of various sizes and ages into my system. Having to switch from the 48-bit ESDI_506.PDR back to regular (LBA-24 + CHS) driver *before* the drive swap is an annoyance (may also cause data loss if the drive reacts unexpectedly to the unrecognised commands).

[Petr]

4. Patched ESDI_506.PDR version of Loew can actually be somewhat dangerous, he does not add any code to check if the drive supports 48-bit LBA, thus the driver may fail if used on a drive that does not support the extended commands. I consider this unacceptable, as I frequently swap drives of various sizes and ages into my system. Having to switch from the 48-bit ESDI_506.PDR back to regular (LBA-24 + CHS) driver *before* the drive swap is an annoyance (may also cause data loss if the drive reacts unexpectedly to the unrecognised commands).

This is not correct. Patched ESDI_506.PDR uses extended commands for accessing sectors above 137 GB barrier, if you have smaller disk, no extended commands are used.

I think that the assumption that drive bigger than 137 GB will support 48-bit LBA commands is reasonable.

Petr

[Marius '95]

I have VIA KT600 (nortbridge) / VT8237 (southbridge). I am using VIA Bus Master (Ultra DMA) PCI IDE Driver v3.20B. This driver replaced ESDI_386.PDR with VIADSK.MPD (ATAPI IDE Miniport Driver v2.0.950.3020). It's much faster than the default driver.

I don't know if this driver supports drives larger than 137GB (I don't have one to test it) but there are two problems with this driver:

- It doesn't support SMART. I tried various utilities but none works.

- If I enable power management, the hard disks enter suspend mode and resume as usual but the CD-RW drive (Teac CD-W524E) stops working after a while. Maybe this driver tries to put CD-ROM drives in suspend mode?

Even with these problems I decided to keep it.

[Eck]

That Via driver does not include 9x support for larger hard drives. I read such in the readme for it. It also shouldn't be used unless using an old 686b Via southbridge chipset with Windows 2000.

Although it installs and runs on other combinations, there are many bugs associated with optical drives and IDE file tranferring, copying, etc.

It also helped with Zip 100 parellel port drives on 9x. But it screwed up other things.

It also cannot be successfully uninstalled once installed. A format is necessary to get things working properly again. They also warn of this in the readme.

[LLXX]

4. Patched ESDI_506.PDR version of Loew can actually be somewhat dangerous, he does not add any code to check if the drive supports 48-bit LBA, thus the driver may fail if used on a drive that does not support the extended commands. I consider this unacceptable, as I frequently swap drives of various sizes and ages into my system. Having to switch from the 48-bit ESDI_506.PDR back to regular (LBA-24 + CHS) driver *before* the drive swap is an annoyance (may also cause data loss if the drive reacts unexpectedly to the unrecognised commands).

This is not correct. Patched ESDI_506.PDR uses extended commands for accessing sectors above 137 GB barrier, if you have smaller disk, no extended commands are used.

I think that the assumption that drive bigger than 137 GB will support 48-bit LBA commands is reasonable.

Petr

I should inspect the code more carefully... but this is from original driver:

L_C0000622:
C0000622 89 43 08		 mov  [ebx+8], eax		  ; store *real* #sectors to read
C0000625 EE		   out  dx, al				; Count register
C0000626 8A 4B 22		 mov  cl, [ebx+22h]
C0000629 D3 E0			shl  eax, cl
C000062B 89 43 0C		 mov  [ebx+0Ch], eax		; #bytes needed or #sectors needed 
C000062E 8B 43 10		 mov  eax, [ebx+10h]
C0000631 8B 8E BB 00 00 00	mov  ecx, [esi+0BBh]
C0000637 F6 81 03 02 00 00 04	 test byte ptr [ecx+203h], 4; LBA_SUPPORT?
C000063E 74 32		jz   short useCHS		  ; no, do CHS instead
C0000640 A9 00 00 00 F0	   test eax, 0F0000000h	; reading past 137Gb?
C0000645 75 2B		jnz  short useCHS		  ; then use CHS instead (this is the error!)
C0000647 66 8B 57 45	  mov  dx, [edi+45h]		 ; LBA low (7:0)
C000064B EE		   out  dx, al
C000064C C1 C8 08		 ror  eax, 8
C000064F 66 8B 57 47	  mov  dx, [edi+47h]		 ; LBA mid (15:8)
C0000653 EE		   out  dx, al
C0000654 C1 C8 08		 ror  eax, 8
C0000657 66 8B 57 49	  mov  dx, [edi+49h]		 ; LBA high (23:16)
C000065B EE		   out  dx, al
C000065C 8A C4		mov  al, ah				; LBA ultra-high? (31:24)
C000065E 8A A6 BA 00 00 00	mov  ah, [esi+0BAh]		; device # (0 or 1)
C0000664 C0 E4 04		 shl  ah, 4
C0000667 0A C4		or   al, ah				; if greater than 137Gb would interfere with Device number!
C0000669 0C E0		or   al, 0E0h			  ; that is why it defaults to CHS for this greater
C000066B 66 8B 57 4B	  mov  dx, [edi+4Bh]		 ; Device register
C000066F EE		   out  dx, al
C0000670 EB 39		jmp  short IssueCommand

This isn't exactly the offending portion but I remember that the driver used the upper byte of the LBA for some other purpose.

Edited May 16, 2006 by LLXX

[RainyShadow]

That Via driver does not include 9x support for larger hard drives. I read such in the readme for it. It also shouldn't be used unless using an old 686b Via southbridge chipset with Windows 2000.

Although it installs and runs on other combinations, there are many bugs associated with optical drives and IDE file tranferring, copying, etc.

It also helped with Zip 100 parellel port drives on 9x. But it screwed up other things.

It also cannot be successfully uninstalled once installed. A format is necessary to get things working properly again. They also warn of this in the readme.

Well, it can be uninstalled without a format, but that's a lot of dirty work.

I accomplished this by extracting some registry entries from another 98SE instalation. ( yay, Nero Tools work again )

Still not sure which exact entries are responsible for that uninstall saga.

REG files included as attachment.

device_regs.rar