Formatting an external drive using different interfaces

[Dave-H]

I have an external 1TB SATA drive in an enclosure, which has two connection interfaces, eSATA and USB.

I am trying to get it working as a backup drive for my two Windows 8.1 Pro computers.

 

Only one computer has an eSATA interface, which works fine and can see the drive no problem.

The other computer, a netbook, can only connect to the drive via USB, which also works fine.

 

The problem is that I can create a full sized primary partition on the drive which can be seen by both machines using the different interfaces, but when I NTFS format the partition on one machine, the other machine doesn't see it as formatted.

If I then format it again on the second machine, the first machine no longer sees it as formatted!

 

Anyone any idea why this should be, surely the type of interface shouldn't make any difference to the way the drive itself is seen? I've also tried using Windows XP (which is also on both machines) to create a logical drive within an extended partition instead of a primary partition (Windows 8.1 doesn't seem to offer this option) but with the same result.

 

The annoying thing is that I'm sure I did have this working with a previous 1TB drive in the same enclosure, which I had to replace as it went faulty. Now try as I might I can't get it to work.

 

Any suggestions gratefully received.

[jaclaz]

Welcome to the wonderful word of Microsoft (and Intel + a bunch of hard disk manufacturers) foolish computing .

 

My crystal ball is as often happens foggy, but it is possible that the internal USB to SATA bridge has decided that your disk is 4096 Bytes sectored while the eSata to SATA bridge (that does not exist and thus is pretty much transparent ) allows the disk (which probably is an advanced format one) to declare itself as 512 bytes sectored (or viceversa).

 

 

Partition and format the NTFS volume through USB connection.

Backup MBR and bootsector on an internal drive

Repartition and re-format the NTFS volume through the eSata connection.

Backup MBR and bootsector on an internal drive

Get Hdhacker (or a similar tool)

http://dimio.altervista.org/eng/index.html

any dd-like tool would do if you are OK with command line, you want firt sector of the \\PhysicalDrive (the MBR) and the first sector of the \\LogicalDrive (the bootsector), using dsfo (part of the DSFOK toolkit) that would be:

dsfo \\.\Physicaldrive<drivenumber> 0 512 C:\myUSB.mbr

dsfo \.\<driveletter>: 0 512 C:\myUSB.bss

then:

dsfo \\.\Physicaldrive<drivenumber> 0 512 C:\myeSATA.mbr

dsfo \.\<driveletter>: 0 512 C:\myeSATA.bss

Compress the 4 files into a .zip archive and attach them and I'll have a look at them.

 

jaclaz

[Dave-H]

Thanks as always jaclaz!

I downloaded and used Dimio's HDHacker.

(Incidentally I've been using his DTaskManager for years now on XP, it's great!)

 

I hope it got the data you need, which is attached.

Both sets of data seemed to be a lot smaller on the eSATA formatted drive than on the USB formatted drive, which seemed odd.

The drive is drive 1 on the netbook (USB), and drive 5 on the main machine (eSATA).

 

All done in Windows 8.1 pro with a full capacity primary partition created.

Cheers, Dave.

DriveData.zip

[jaclaz]

Yep, though foggy, the thingy worked fine.

 

You can double-check with fsutil, but you are in this situation :

http://msdn.microsoft.com/en-us/library/windows/desktop/hh848035(v=vs.85).aspx

EITHER the stupid disk drive (for NO reason whatever since it is only 1 Tb) is "Advanced Format" OR the stupid enclosure (also for NO reason whatever) translates the disk sector size to 4096 bytes from the "native" or "converted" 512.

 

Which EXACT hard disk make/model is it?

 

jaclaz

[Dave-H]

Thanks jaclaz.

It's a Western Digital WDC WD10EZEX-00BN5A0 drive.

That all looks very complicated, but might indeed be the explanation as to why it's a problem now that it wasn't before, as my previous drive was a few years old and therefore almost certainly wouldn't have been "advanced format".

Do you think I should try doing the format with a different allocation unit size.

I had been using the default, which appears to be 4096.

[jaclaz]

NO, the issue is different, and I presume at a lower level than I (or you) would like it to be.

 

Please do check with fsutil as in the given link when connected through the one (or the other) interface and post results.

 

My guess at the WHY this is happening, I suspect that there has been a "defect of communication" (or maybe simple stupidity).

The good MS guys, before getting on the senseless Intel originated EFI/UEFI madness, had the idea of solving the 2.2 Tb limit the "right" way, i.e. by increasing the physical sector size from 512 to 4096.

The good guys from the various hard disk manufacturing companies were happy about this as - as a matter of fact - their disk drives would have a slight advantage.

BUT the same good MS guys did not add (or did nto add in a timely enough manner) "native" support for booting and accessing  4096 sectored devices, adding access to it ONLY for USB connected disks.

The good hard disk manufacturers then needed to invent the AF drives, drives that are 4096 bytes sectored but that are exposing themselves through a "translation interface" as 512 bytes devices so that the same disk could be used internally and inside an external USB enclosure.

In all this some other good guys (the ones that manufacturer USB external enclosures) thought that their enclosure should be able to handle larger than 2.2 Tb disk and since the way set by the good MS guys was to use 4Kb sectored devices, and support for accessing them (but not for booting from them) was available decide to set the USB to Sata bridge to make the translation, i.e. to expose the disk as 4 Kb only.

 

The WD10EZEX of 1 Tb uses according to specs 1,953,525,169 sectors, so definitely it is a "pure" (as it should be BTW) 512 bytes sectored device.

 

It is possible that the previous disk was an AF drive that allowed "by itself" the 4096 bytes per sector "emulation".

 

Right now I have no idea if a workaround for your situation exists and/or if it exists how many attached strings it may have.

In theory and with a couple tricks it could be possible to correct the partition table and the boosector when the disk is accessed through one (or the other) interface by running a batch program each time,  in theory the NTFS filesystem operates on clusters, and since the clusters are anyway 4096 bytes, once the size of sector and the sector per cluster values have been corrected the filesystem should be accessible and work exactly the same, I am more worried about possible incompatibilities with lower level tools like partition imagers, or similar.

 

jaclaz

[osRe]

Didn't look at the MBRs, and quite probably unrelated, but worth a try. Someone reported the drive is not working for him on a motherboard controller, and the solution was to put a jumper on the drive between pins 7 and 8:

http://www.techbuy.com.au/p/200204/HDD_INTERNAL_SATA-II_DRIVES/Western_Digital/WD10EZEX.asp

As far as I can tell this only offsets the sector alignment to fix performance problems when using XP's default partition alignment, but who knows.

jaclaz: What did you see in the MBRs?

AF is the general name for 4K physical sector drives. The 512-byte sector emulation is called 512e, and it's not only because of Microsoft but because of software everywhere. It's not a bad idea.

Edited January 3, 2015 by shae

[Dave-H]

Thanks shae, I'll try that jumper if all else fails!

 

I've run fsutil on both systems, and the results are attached.

As expected, they are very different!

 

eSATA fsutil output.txt

USB fsutil output.txt

 

 

[jaclaz]

jaclaz: What did you see in the MBRs?

 

What was expected .

The partition table when ESATA connected is that of a 512 bytes sector device, the partition table when USB connected is that of a 4096 bytes sector device, see the attachments:

 

The values in the eSATA are 8x the ones in the USB one.

eSATA:

The first (and only) partition starts at offset 2048*512=1048576 bytes and extends for 1953519616*512=1000202043392 bytes

USB:

The first (and only) partition starts at offset 256*4096=1048576 bytes and extends for 244189952*4096=1000202043392 bytes

 

As said, the specific model, according to its specsheet:

http://www.wdc.com/wdproducts/library/SpecSheet/ENG/2879-771436.pdf

is NOT an AF drive and NOT a 512e drive, it is a simple, normal, "pure" 512 bytes sectored device, and this has been just confirmed by the results of the fsutil info Dave-H posted.

To be confirmed/checked.

 

 

jaclaz

PTVIEWSATA.html

PTVIEWUSB.html

Edited January 3, 2015 by jaclaz

[Dave-H]

So, is there anything that can be done about this?

Is it worth trying the jumper, or is that "clutching at straws"?

Will trying to set the drive up using XP be any better?

I have tried that already of course, with pretty much the same result.

[jaclaz]

The page shae found is a "generic" one, the specific disk is - at least from the specifications from the manufacturer - a "native" 512 bytes bytes disk, it's sectors are 512 bytes in size.

To be confirmed/checked.

 

It is a "rare" case, actually the first one I find of this kind, you have a device that to the OS is 512 bytes sectored if connected through eSata (i.e. "directly") and 4096 bytes if connected through the USB bridge.

 

The "real" solution (very unlike to be found) would be to re-program the controller in the USB bridge to avoid the translation, see also (though not really related):

http://reboot.pro/topic/20228-boot-from-4k-sectors-and-in-particular-usb/

 

Maybe, and I have to underline maybe, it is possible through rewriting every time you change the interface/connection the MBR and the bootsector to have a working filesystem.

 

Apart changing those, the real issue is that $MFT records on 512 bytes sectored devices are 1024 bytes, whilst on 4k sectored devices they become as well 4k (minimum addressable element):

http://blogs.msdn.com/b/ntdebugging/archive/2011/06/28/ntfs-and-4k-disks.aspx

 

What I don't know (and I may need you to experiment with this) if once a NTFS volume has been formatted with 4k sectors and the few bytes are changed in the MBR and bootsector to work on 512 byte sectored the 4k size of the $MFT record is kept or if it will "revert" to 1024 bytes and/or if there is a setting *somewhere* different from the values in the bootsector that governs this $MFT record size.

 

The other possible solution (also unlike to be found unless we find some good guy with excellent programming capabilities) would be a "filter driver" to install on the machine that uses the eSata connection to make the disk appear as 4 kb sectored as it is when connected through the USB.

It is possible that such a filter driver already exists, but if it does I know nothing about it.

 

jaclaz

Edited January 3, 2015 by jaclaz

[osRe]

The WD PDF above lists the drive as AF. See the "Advanced Format" row.

The jumper thing probably won't help, but it's easy to try, so why not? It's not like you have a problem formatting the drive, for the time being.

And apparently this is a common problem:

http://superuser.com/questions/719844/do-hard-disk-drives-turn-on-512e-512byte-emulation-of-4k-sectors-as-needed-dep

http://superuser.com/questions/410606/logical-sector-size-changes-depending-on-whether-it-is-attached-via-usb-or-direc

http://superuser.com/questions/463952/is-it-possible-to-set-the-logical-sector-size-of-a-usb-hdd

Stupid hardware/software/standards.

Edited January 3, 2015 by shae

[jaclaz]

The WD PDF above lists the drive as AF. See the "Advanced Format" row.

The jumper thing probably won't help, but it's easy to try, so why not? It's not like you have a problem formatting the drive, for the time being.

I stand corrected.

 

I don't know.

 

What Dave-H got was (eSATA) which is the behaviour of 512n or 512 native" :

 

Bytes Per Sector : 512

Bytes Per Physical Sector : 512

 

 

What he got (USB):

 

Bytes Per Sector : 4096

Bytes Per Physical Sector : 4096

 

 

How an AF (512e) drive should be detected:

http://msdn.microsoft.com/en-us/library/windows/desktop/hh848035(v=vs.85).aspx

 

Bytes Per Sector : 512

Bytes Per Physical Sector : 4096

 

 

So the jumper is to emulate the 4096 Bytes Per Physical Sector?

 

I have believed till now that the AF format drives were "native" 4096 Kb and that in the case of 512e the emulation was "downwards", is it possible that the good WD guys have a "full" 512 bytes emulation?

 

jaclaz

Edited January 3, 2015 by jaclaz

[osRe]

No, the jumper just adds +1 sector offset to make XP's partitioning scheme create the first partition starting at sector 64. But this doesn't explain why that mobo didn't boot or recognize the drive otherwise (as reported in a random thread I read).

I just checked an AF drive I have, currently in an old eSATA/USB enclosure based around Sunplus SPIF225A. It works the same with both connections. Through USB it appears as 512 bytes/sector physical. Though eSATA, well, I'll have to find a utility to report that on XP.

Edited January 3, 2015 by shae

[dencorso]

The solution is to buy WD Caviar Black HDDs of the FAEX kind, which are the last non-AF still available (notice there exists 3 and 4 TB versions of them, too!!!)... and then sell, or use only through USB, the Caviar Blue AF 1TB Dave-H already has.  Sorry!  Of course, that's just my 2¢. Myself, I'm hoarding WD2002FAEX, WD1502FAEX and WD1002FAEX HDDs, in order to be, let's say (literally)... "Zukunftsfähig"

WD Caviar Black (512 or AF) 2014.pdf

WD Caviar Black (512 or AF) 2013.pdf

WD Caviar Blue (512 or AF) 2013.pdf

WD Caviar Black (512 or AF) 2012.pdf