Problems with MBR hard disk 5 TB

[Cixert]

I had discussed several problems with this 5 TiB Seagate hard drive in this post.

https://msfn.org/board/topic/183934-seagate-external-hard-drive-is-xp-incompatible/

I have kept this 5 TB harddisk unused until now.
I have proceeded to copy my 4 TB hard drive to this 5 TB hard drive by expanding the partitions.
I have connected to the computer with the only Logilink USB adapter model that allows working with hard drives larger than 4 TiB.
Oldest USB Logilink AU0028 (without letter "A") work with MBR hard drive with 5 TB, newer Logilink adapters are limited to 2 or 4 TB depending on the installed firmware.

I HAVE SEVERAL PROBLEMS:
1-I have proceeded to create partitions with the Windows Disk Manager and it does not work correctly. It only allows me to create full disks with no partitions or partitions up to 63 MiB. When I move the box to expand size it gets stuck at 63 MiB. If I decrease the size it starts to show negative -100 MiB, -400 Mib, etc.

2-OK, I correctly partitioned the disk with MiniTool Partition Wizard 10.3 (higher versions don't work well with any disk).
So I have 3 primary and 3 extended partitions.
I proceed to copy the old disk partition by partition to the new disk with R-Drive Image.

Partition 1 original 1.03 TiB on NTFS cluster 4 KiB I want to paste in new partition 1.32 TiB NTFS 8 KiB.
This seems to work without problems.

Partition 2 original 759 GiB on FAT32 cluster 8 KiB I want to paste in new partition 1.17 TiB on FAT32 cluster 16 KiB.
The operation says to finish without problems, but when I go to My PC the file explorer says that instead of 1.17 TiB the partition measures 182 GiB. When entering the content the folders are visible but when trying to access the files these are faulty.
When I go to Windows Disk Manager it says that the partition is correct and that it measures 1.17 TiB.
When I examine the disk with any partitioning program it says everything is ok and the partition is 1.17 TiB, however My PC still says the partition is 182 GiB.

what is the problem, why my computer doesn't show the partition correctly after copying with R-Drive Image?

The old 4TB drive has the first partition in sector 63 and according to Minitools it is misaligned.
I have aligned the new 5 TiB disk with MiniTool before starting the copy and it has put the first partition in sector 256.
Is it correct for a partition to start at sector 256?
I am checking the size of the clusters and I see that after proceeding with the copy the original cluster size has been respected, both in the NTFS partition and in the FAT32 partition. That is, R-Drive Image has not copied the partitions with the new assigned cluster size.
I have tried to solve the problem with larger cluster sizes but when making the copy the original cluster size is assigned.

3-OK, I proceed to make more copies of FAT32 partitions with large sizes (600-800 GiB) without problems.
The problem seems to be when I exceed 1 TiB in size in FAT32.

How do I get My PC to correctly recognize the partition?
Is it a size limit problem on 1TiB FAT32 or is the problem in the R-Drive Image software?

Edited July 23, 2023 by Cixert
link ok

[jaclaz]

The link you posted is somehow invalid, this is the right one:

https://msfn.org/board/topic/183934-seagate-external-hard-drive-is-xp-incompatible/

Nothing has changed since then.

If your disk is exposed as being 4K sectors by the USB adapter, then it should (might) work with MBR style, if it is exposed as being 512 bytes sector you will be limited to 2.2 TB in MBR.

TB sized partitions FAT32 are "pure folly", anyway the cluster size for volumes larger than 32 GB should be 32 KB:

https://support.microsoft.com/en-us/topic/default-cluster-size-for-ntfs-fat-and-exfat-9772e6f1-e31a-00d7-e18f-73169155af95

Again, see a few posts starting from here:

https://msfn.org/board/topic/118623-clone-easily-windows-98-and-xp-in-the-same-computer/page/8/#comment-866527

(link already given in the other thread)

On NTFS it makes no sense to use larger than 4KB clusters, that are good up to 16 TB.

If you created a partition 1.17 TB, that is the size of the partition, but this tells nothing about the size of the volume inside it, disk manager and partitioning tools look at the size of the partition, explorer and other file managers access the filesystem, that since you "pasted" it may have a different size or even come from a filesystem on a disk where the 512 bytes sector was in use..

Sector 256 may be ok (if the disk is exposed as 4kb sectors) as it is the default alignment to 1 MB used in Vista and later.

There is a known risk when using XP disk manager on a disk where volumes inside extended partition are created with a later system (with the 1 MB alignment), this should not apply to volumes inside extended partition created under XP with disk manager, but they will NOT be aligned to the MB, if they are, you risk losing all of them at the first time you use disk manager to change *anything* on the disk.

Cluster size is a characteristic of the filesystem, you can only set it when formatting a volume, if you copy a whole partition the original cluster size will obviously be maintained.

It has to be seen what happens when you resize (expand) a volume, a "normal" tool should stop at  the max cluster size allowed for the size of the volume, without changing cluster size as changing it would essentially mean to rewrite the whole FAT tables or the $MFT.

jaclaz

 

 

 

[Cixert]

12 hours ago, jaclaz said:

 

On NTFS it makes no sense to use larger than 4KB clusters, that are good up to 16 TB.

Cluster size is a characteristic of the filesystem, you can only set it when formatting a volume, if you copy a whole partition the original cluster size will obviously be maintained.

It seems to be a cluster size issue. The physical cluster is 4 KiB.
I have converted the partition inaccessible from My PC from 8 KiB cluster to 64 KiB cluster and now it is accessible from My PC and the files are displayed correctly. From what it seems it is a cluster size problem. Unfortunately MiniTool Partition Wizard only gave me the option to convert the cluster size to 2 KiB or 64 KiB, not 16 or 32 KiB. However, it does allow me to format it with 16 KiB.
I want to clone my old 4TB MBR HDD to my new 5TB HDD by expanding partitions and cluster size.
Does any version of R-Drive allow this operation?  Does any other program allow you to perform this operation?
Because otherwise it is not possible to clone disks with copies of incremental size.
That is, in the case of the problematic partition, the original one measures 759 MiB with FAT32 and a cluster size of 8 KiB. I want to copy this partition to the new disk with a size of 1.17 TIB FAT32 and a cluster size of 16 KiB. But when cloning it with R-Drive Image, it formats the new partition with the cluster size of the original partition, finally making the data inaccessible.

Maybe I am wrong, but I think I have read that in NTFS with a cluster size of more than 4 KiB it is not possible to encrypt the drive. I don't want a virus to enter to encrypt my data. That's why I want to put in NTFS a cluster size of 8 KiB.
In any case, won't this partition go faster with an 8 KiB cluster?

Edited July 19, 2023 by Cixert

[jaclaz]

There is no such thing as Physical cluster, that is the Physical sector size.

It is possible that the original disk exposed 512 bytes sectors.

Then the volume was formatted, with the whatever cluster size that was applied to the filesystem, was a multiple of the (physical) sector, this is the actual way the cluster size is expressed in the bootsector, the field is called "Sectors per cluster" and (as an example) the common 4096 bytes sized cluster is expressed as 8 (because 8*512=4096).

If you simply copied the volume, and nothing adjusted that value the filesystem will become invalid, as that same 8 will be interpreted as 8*4096=64 KB.

When you used that tool to "convert" the cluster size it evidently changed and fixed that value (+ possibly a few others as needed to make a valid filesystem).

I don't know of any tool that will clone a volume and adjust these values, actually it won't be a clone at all and there remains some risks with converting these volumes.

Besides, at least in the case of NTFS, this kind of conversion would equate (if possible at all) to a re-write of the filesystem, a "normal" $MFT entry on a NTFS filesystem on a 512 bytes disk is 1024 bytes while a $MFT entry on a NTFS filesystem on a 4095 bytes disk will be 4096 bytes.

In a nutshell, you cannot (and shouldn't) clone a volume from a 512 bytes disk to a 4096 bytes disk.

No idea whether the (built-in EFS) NTFS encryption has any limit on cluster size, but it is not like, even if true, the hypothetical virus would surely use the standard NTFS encryption, and not any other algorithm/encryption scheme.

jaclaz

 

[Cixert]

14 hours ago, jaclaz said:

There is no such thing as Physical cluster, that is the Physical sector size.

It is possible that the original disk exposed 512 bytes sectors.

Then the volume was formatted, with the whatever cluster size that was applied to the filesystem, was a multiple of the (physical) sector, this is the actual way the cluster size is expressed in the bootsector, the field is called "Sectors per cluster" and (as an example) the common 4096 bytes sized cluster is expressed as 8 (because 8*512=4096).

If you simply copied the volume, and nothing adjusted that value the filesystem will become invalid, as that same 8 will be interpreted as 8*4096=64 KB.

When you used that tool to "convert" the cluster size it evidently changed and fixed that value (+ possibly a few others as needed to make a valid filesystem).

I don't know of any tool that will clone a volume and adjust these values, actually it won't be a clone at all and there remains some risks with converting these volumes.

Besides, at least in the case of NTFS, this kind of conversion would equate (if possible at all) to a re-write of the filesystem, a "normal" $MFT entry on a NTFS filesystem on a 512 bytes disk is 1024 bytes while a $MFT entry on a NTFS filesystem on a 4095 bytes disk will be 4096 bytes.

In a nutshell, you cannot (and shouldn't) clone a volume from a 512 bytes disk to a 4096 bytes disk.

No idea whether the (built-in EFS) NTFS encryption has any limit on cluster size, but it is not like, even if true, the hypothetical virus would surely use the standard NTFS encryption, and not any other algorithm/encryption scheme.

jaclaz

 

I will continue testing.
I'll try to copy the old drive to the new drive by creating an image of the old one and then restoring it to the new one.
Both disks have the sector size at 4096.
At the moment, what I can confirm is that Windows Disk Manager does not work with 5TB hard drives, whether the partition starts at sector 63 or sector 256. It is not capable of creating partitions.

Edit:
I am testing with a 300 GB disk and the box does not let me change the size of the partitions either. I do not know what is happening, I have already tried on several computers.
However, I can create partitions if I manually type the size figure in megabytes.

Edited July 20, 2023 by Cixert

[jaclaz]

12 hours ago, Cixert said:

Both disks have the sector size at 4096.

And this in itself means nearly nothing.

Most recent disks have 4096 bytes sectors.

Then most of them are set as 512e, i.e. while they do have 4096 bytes sectors, they expose to the partitioning/formatting tool 512 bytes sectors.

So called "Native 4K" are relatively few.

Then comes into play (in case of external disks) the enclosure or the "bridge" in it (be it USB or other).

Some of these make no translation, some translate the disk from 512 bytes sector (512e or not) to 4096 bytes, some (hopefully only a few) may even be intended to translate Native 4k to 512 bytes sector..

It is complicated.

You could (should) analyze the bootsector of those volumes to understand how they were formatted, checking the fields "Bytes per sector" and "Sectors per cluster" you will be able to make sure what the sector size (at the time the volume was originally formatted) was and the actual (again at the time it was formatted) cluster size.

jaclaz

[Milkinis]

21 hours ago, jaclaz said:

So called "Native 4K" are relatively few.

is 512e/4Kn the same thing and 512 only ?

the first one seems to be able to emulate either of the sectors size.

[Cixert]

There are things that don't add up for the problem to be due to a cluster size limitation.

1-So, why is the data physically there and after converting the partition to a cluster size of 64 KiB now the data is accessible from My PC?

2-I have made the copy again to check if other systems are capable of accessing the data with an 8 KiB cluster.
And no, it's the same for everyone. I have tried:
-Total Commander
-Windows 2000
-Windows Xp
-Windows 10
Another FAT32 partition with 923 GiB and 32 KiB cluster size is perfectly accessible. And partition with 87 GiB and cluster 8 KiB also accessible.

I have right now:
Primary partition 1: 1.32 TiB NTFS cluster 4KiB
Primary partition 2: 1.17 TiB FAT32 cluster 8KiB
Primary partition 3: 87 GiB FAT32 cluster 8KiB
Extended partition 1: 683 GiB FAT32 cluster 32KiB
Extended partition 2: 389 GiB FAT32 cluster 32KiB
Extended partition 3: 923 GiB FAT32 cluster 32KiB

What cluster size limit are we talking about?
I have pointed out that with a cluster size of 512 bytes the total limit of the sum of all MBR partitions is 2 TiB, but if the virtual size of the cluster is increased, the partitions can be larger. I know @jaclaz sees it differently.
 

Then there is the maximum number of files a FAT32 partition can have. But it is that I am copying the same files that right now are working on My PC with a cluster size of 8 KiB. The only thing that changes is the size of the partition, which is now bigger. The number of files is the same.
In total I have 281,584 files in 29,792 folders.
I read that the maximum number of FAT32 files with 8 KiB cluster is 67,043,325

Edited July 21, 2023 by Cixert

[j7n]

Can you connect your disk directly to the motherboard to eliminate the USB adapter? Use a controller driver that supports large disks, and create normal GPT partitions.

The limit isn't impacted by the cluster size selected in the file system because it happens on the physical sector level. If you want to change the sector size of your partitions, then you must create new partitions and copy files onto them individually. Do you have some complex licensed software onto these partitions that requires them to be preserved verbatim? Would that software be happy to run/boot from a dodgy USB adapter?

You could try to copy the entire source disk starting from sector 0, including its partition table, and don't employ any partitioning tools whatsoever, but that would keep the old aligment of 63 obviously. Then add another partition to the end or extend the last one to fill up the remaining space. When you created "placeholders" for the old partitions, did their size match down to the sector? If you specified their size in megabytes, they might not be the exact same size.

[jaclaz]

Final statements:

Sector size is determined by the manufacturer of the hard disk and cannot normally be changed (some hard disks, through a proprietary manufacturer tool may allow to switch from 512e and 4kN or viceversa).

Some USB adapters/bridges/enclosures might translate sector sizes (from 512 to 4096 or - maybe - from 4096 to 512), in some cases it may be possible to change the behaviour of the chip inside the bridge through a proprietary manufacturer tool.

Cluster size is a characteristic of the file system and is normally determined/chosen at the time of formatting a volume.

Its value is determined by two fields in the BPB (Bios Parameter Block), usually called "Bytes per sector" and "Sector per cluster", multiplying them you obtain the cluster size, examples:

Bytes per sector=512
Sectors per cluster=8
Cluster size=512x8=4096 bytes

Bytes per sector=4096
Sectors per cluster=1
Cluster size=4096x1=4096

In the MBR there is NO data related to clusters[1], ONLY about sectors (as seen by the OS), hence the limit of around 2.2 Tib for 512 bytes sectors becomes, when using 4096 bytes sectors (8x), roughly 17 TiB (8x)[2]

 

@Milkinis

A 512 (old, traditional) disk has internal physical sector size of 512 bytes AND exposes 512 bytes externally.
A 512e disk has an internal physical sector size of 4096 bytes BUT exposes 512 bytes externally.
A 4kN disk has an internal physical sector size of 4096 bytes AND exposes 4096 bytes externally.

jaclaz

 

[1] AGAIN clusters are determined in the file system, the MBR knows NOTHING about the file system, thus NOTHING about cluster size
[2] and this is not how jaclaz "sees it differently", it is how it is

[Cixert]

13 hours ago, j7n said:

Can you connect your disk directly to the motherboard to eliminate the USB adapter? Use a controller driver that supports large disks, and create normal GPT partitions.

The limit isn't impacted by the cluster size selected in the file system because it happens on the physical sector level. If you want to change the sector size of your partitions, then you must create new partitions and copy files onto them individually. Do you have some complex licensed software onto these partitions that requires them to be preserved verbatim? Would that software be happy to run/boot from a dodgy USB adapter?

You could try to copy the entire source disk starting from sector 0, including its partition table, and don't employ any partitioning tools whatsoever, but that would keep the old aligment of 63 obviously. Then add another partition to the end or extend the last one to fill up the remaining space. When you created "placeholders" for the old partitions, did their size match down to the sector? If you specified their size in megabytes, they might not be the exact same size.

-I prefer to use MBR disks for compatibility with Windows 2000.
-I don't have any additional software that affects the file system.
-The old disk partitions start at sector 63 and are misaligned. But it does not affect the copies on the new disk, since I have previously aligned it and when making the copies partition by partition the alignment of the new disk is maintained

3 hours ago, jaclaz said:

Final statements:

Sector size is determined by the manufacturer of the hard disk and cannot normally be changed (some hard disks, through a proprietary manufacturer tool may allow to switch from 512e and 4kN or viceversa).

Some USB adapters/bridges/enclosures might translate sector sizes (from 512 to 4096 or - maybe - from 4096 to 512), in some cases it may be possible to change the behaviour of the chip inside the bridge through a proprietary manufacturer tool.

Cluster size is a characteristic of the file system and is normally determined/chosen at the time of formatting a volume.

Its value is determined by two fields in the BPB (Bios Parameter Block), usually called "Bytes per sector" and "Sector per cluster", multiplying them you obtain the cluster size, examples:

Bytes per sector=512
Sectors per cluster=8
Cluster size=512x8=4096 bytes

Bytes per sector=4096
Sectors per cluster=1
Cluster size=4096x1=4096

In the MBR there is NO data related to clusters[1], ONLY about sectors (as seen by the OS), hence the limit of around 2.2 Tib for 512 bytes sectors becomes, when using 4096 bytes sectors (8x), roughly 17 TiB (8x)[2]

 

@Milkinis

A 512 (old, traditional) disk has internal physical sector size of 512 bytes AND exposes 512 bytes externally.
A 512e disk has an internal physical sector size of 4096 bytes BUT exposes 512 bytes externally.
A 4kN disk has an internal physical sector size of 4096 bytes AND exposes 4096 bytes externally.

jaclaz

 

[1] AGAIN clusters are determined in the file system, the MBR knows NOTHING about the file system, thus NOTHING about cluster size
[2] and this is not how jaclaz "sees it differently", it is how it is

So, what is the problem that the partition is not accessible and why by software changing the FAT32 cluster size to 64 KiB the data is now accessible?
Do you think that if I copy the data in this partition instead of in FAT32 in NTFS with size 4 KiB cluster the data will be accessible? I'm going to check it out.

Thanks

 

Edited July 22, 2023 by Cixert

[Milkinis]

23 hours ago, jaclaz said:

A 512 (old, traditional) disk has internal physical sector size of 512 bytes AND exposes 512 bytes externally.
A 512e disk has an internal physical sector size of 4096 bytes BUT exposes 512 bytes externally.
A 4kN disk has an internal physical sector size of 4096 bytes AND exposes 4096 bytes externally.

 

how do you determine the number of sectors per cluster ? so it's either 1 or 8 ?

the 512e thing is what got me into the confusion.

so the 512e is a single sector/cluster of 4096 bytes but old Windows editions under W8 will see 8 sectors of 512 instead ?

I have just checked out an old Seagate HDD 

cmd = Fsutil fsinfo ntfsinfo x:

bytes per sector = 512

bytes per cluser = 4096

 

 

 

 

 

[Milkinis]

On 7/20/2023 at 12:21 PM, jaclaz said:

So called "Native 4K" are relatively few.

you are correct. 

the few ones I found spin at 7200rpm only and are intended for data centers data storage which means they do cost twice as much

even so there are either versions 512e or 4Kn for SATA and SAS.

 

 

[jaclaz]

No. AGAIN clusters have NOTHING to do with disks (and a lot to do with volumes and filesystems).

See here:

https://kb.romexsoftware.com/en-us/3-general/51-512n-512e-and-4kn-drives

https://learn.microsoft.com/en-us/troubleshoot/windows-server/backup-and-storage/support-policy-4k-sector-hard-drives

You want to look at values of

Bytes per sector:

Bytes per Physical sector:

and nothing else.

It is  confusing (and very unfortunate) that the built-in command in Windows to get these values is actually a tool dedicated to gather info on the volume/filesystem (and actually reserved to NTFS only).

There seems to be a "more proper" way to look at this info, that works on the disk (not on the NTFS volume) using Powershell, but only on WIndows 8 and later:

https://superuser.com/questions/1040865/how-can-i-get-the-physical-sector-size-of-a-drive-that-doesnt-have-any-recogniz

Get-Disk | Format-List

In Linux, fdisk -l should do.

A 512n disk will be seen by any OS as having 512 bytes/sector. (internally it uses 512 bytes sectors)

A 512e disk will be seen by any OS as having 512 bytes/sector.(internally it uses 4096 bytes sectors that are "fractioned" into 512 bytes ones transparently)

A 4kN disk will be seen by any OS as having 4096 bytes sector. (internally it uses 4096 bytes sectors)

jaclaz

 

[Cixert]

The real size of the sector can be checked on W2000 & on XP with Eassos Disk Genius.
All my hard drives +2TB have a phisical size 4096 bytes sector.
I have also verified this from Windows 10 with Fsutil Fsinfo Ntfsinfo X:

When in the past I did backup copies on a hard drive 3 TB and on a 4TB hard drive I studied the limits of Fat32 partitions.
Unfortunately I "do not" remember that and I do not find real information on the Internet right now.

What are the limits of FAT32 partitions?
There is a limit, that's true.
But, I don't know if this is a limit on the cluster number or in the number of files.
I think I remember that cluster size must be increased as partition size increases.
My current situation is the following with the problematic partition, copying 759 GiB of the old Fat 32 cluster 8 kib partition to a new partition with 1208 GiB.

I copy with R-Drive Image an image in another partition.
I restore the image data in the problematic partition formatting this way:
-NTFS 4 kib: No problem.
-FAT32 8KiB: Windows reports an incorrect partition size and many files are not accessible.
-FAT32 16 KIB: The data is correctly restored in appearance but these are the problems:

1-Windows 2000 correctly informs the size of the partition on my PC but gives an incorrect size in the number of files and folders on properties after selecting all folders. This happens both in the old partition (cluster 8 KiB) and in the new partition (cluster 16 KiB).




Windows XP does report the number of files and folders correctly

2-Windows 2000 It is blocked after executing the Chkdsk order without parameters.
In the new partition. When I restart the Windows 2000 system, it is not able to check if the file system is correct, in the new partition (16 kib) and Windows 2000 does not start. The old partition (8kib) is correctly proven.
But both partitions cannot be proven in verifying errors on tools of My PC.

In Windows XP the system starts without problems, but on this new computer and another that I have tried Windows XP does not check the USB hard drives file system when starting the system. On the old computer that I had before the old hard drive the partitions were checked when the Windows XP system started. Why are they now not checked on new computer?
Windows XP is also not able to check partition errors with the tool that is on my PC.

In summary, after copying 758 GiB of a FAT32 partition with cluster 8 KiB in a 1208 GiB partition with 16 KiB cluster there are problems, less than if the new partition had cluster 8 KiB but exist.
What is the origin of the problems?
What are the real limits of FAT32 partitions?

According to Wikipedia, FAT32 partions have a 2 TB limit with a 32 kib cluster size.
If this is true, then what is the limit of a 1 TB partition?
In addition, Wikipedia says that the maximum number of files in a FAT32 partition is 268,173,300 files with cluster 32 kib.
I have 278,275 files in old partition of 759 Gib with cluster 8 Kib, working so far without problems.
But with the same files in the new partition of 1208 GiB with cluster size 16 kib there are problems.

Old partition 759 GiB on Disk Genius


New partition 1208 GiB on Disk Genius

.

.

.

 

Edited August 2, 2023 by Cixert