Hosed NTFS cluster alignment?

[Ergaster]

What's the easiest way to "shift the ground" under a mis-aligned NTFS file system?

After a bad interaction with Linux partitioning software, a properly aligned NTFS 4k_cluster storage-only partition on an advanced format drive got hosed, and Vista re-built the file system seemingly from scratch. There are scant to no traces of any other valid file system except an abandoned NTFS boot sector at 63 (!) which references a $MFT that is literally all FF'd-up.

The valid boot sector is at 4096, and there is no way back to "what-was" within the current file system. The data is all there, the partition checks out perfect, but the references to fragment runs are -all- 1985 sectors early! Obviously, that doesn't line up with the clusters or the files.

My first thought is - can I just copy the NTFS boot/backup sector, $MFT/metadata files, and folder files, up 1985 sectors (holding their relative positions) and update the MBR? Is there more to this? There's a bunch of empty space before and after the data in the partition. Only need to copy essential infrastructure to mount and copy partition contents, or not even mount it and use Ghost from DOS to image a properly aligned NTFS.

Could change the cluster size to 512 and recalculate every fragment run (hundreds).

I have the output of Microsoft's old NFI tool for this partition, giving me logical sector-based runs for every file. Working from that, I can manually offset and concat each fragment of a given file with a disk editor, but that's a last resort.

Tips or tools to make this LESS complicated?

[jaclaz]

I am not sure to get what the problem is.

If the data is accessible, the easiest would be to backup the data, re-format properly and restore.

The $MFT and $MFT mirror can be *anywhere* as long as their location is correct in the bootsector.

The bootsector is first sector and cannot be moved.

The bootsector backup is last sector (actully outside the filesystem) and cannot be moved.

Changing contents of the $MFT is tricky business (and simply FORGET about doing it "manually"),

Can you expand on the actual problem?

jaclaz

[Ergaster]

Ok, so you're going to read a book. The cover looks fine, the table of contents looks fine, the book has all its pages, and everything is ready to go, so you go to a chapter on the page where it's supposed to start, and the chapter doesn't start there! You're looking at the last several pages of the previous chapter, and have to turn several pages before you find the chapter heading of what you wanted to read.

The partition in question is like that, except it uses logical sectors and is fragmented.

You have to add 1985 logical sectors to everything to find what you expected to find. But all the fragment runs in a given file will truncate 1985 sectors early. You can run past the truncation with a disk editor and see the rest of the fragment, which is more obvious if there is free space or something very different after it. I went around and randomly checked several dozen files all over the partition, and they're all affected the same way.

Nothing is "wrong" with the $MFT from a structural standpoint, so you can check the disk with any program and find no problem. It just points to the wrong places in the user-data. It doesn't point to the wrong places in its own infrastructure, such as folder lists and its own metafiles.

[jaclaz]

OK, maybe I am getting an idea of the situation (though I am not entirely sure HOW that could have happened ).

So, given that there are THREE parts of the filesystem (let's say conventionally "thirds"), first "third" physically before the $MFT and second "third " after the end of it and before the $MFTmirror, and third "third" after it till the end of the volume.

Are files in ALL "thirds" displaced by the same amount of sectors?

1985*512=1016320 which is at first sight a "strange" number

1985+63=2048 and 2048*512=1048576 sounds more "like it".

Let's start thinking in terms of "Lego blocks".

We have 7 "main" blocks:

1. bootsector (16 sectors in size, but the ONLY meaningful one for this is first one)
   
2. "first third of data"
   
3. $MFT (size depends on overall size of volume) - normally this is at LCN 786432 dec
   
4. "second third of data"
   
5. $MFT mirror (first 4 records of $MFT if I recall correctly) - normally this in the middle of the volume i.e. Size_in_sectors/cluster_size/2
   
6. third "third" of data
   
7. backup bootsector (after the end of the volume, i.e. outside the filesystem but within partition space)
   

All addresses are relative to beginning of the volume, so (and I am not sure to have it right) if you have a file that should be at Absolute sector (say) 10,000, you instead find it at Absolute sector 8,015, you need to have the beginning of the volume (actually the bootsector or block #1 above) 1,985 sectors BEFORE.

If it's the other way round, i.e. you find the beginning of the file that should be at Absolute sector 10,000 on Absolute sector 11,985 you need to have the beginning of the volume 1,985 sectors LATER.

In other words, you need to check:

1. the current MBR entry for that partition
   
2. the current bootsector that you can find at the first sector of address specified in the above
   
3. the backup bootsector that you can (possibly) find on last sector of the partition, i.e. in last sector of address specified in the above
   

Then:

1. Verify that bootsector and backup bootsector are the same.
   
2. Verify that at the "other" theoretical address, i.le. if "current" is at Absolute sector 63, check at Absolute Sector 2048 or, viceversa, there is another bootsector (with different data).
   
3. Correct the MBR to set the partition start at the correct offset.
   

If you can post these sectors (make a RAW copy of each of them, name them meaningfully, compress them together in a .zip file) I can have a look at them and give you some more "exact" help.

A tool suited for this kind of checks is tiny hexer (optionally making use of my "Structure viewers" for it), reference:

http://reboot.pro/8734/

but you can of course use any disk editor you are familiar with.

jaclaz

[Ergaster]

There were 3 boot sectors - two matching ones at either end of the partition, and one at 63 which pointed to an offset that was all FF FF FF FF.... no other $Mft could be found.

The de-coupling of the file system and the data means that the new/only $Mft and all of its external attributes are not cluster-aligned with the "original" data storage area, so it has to move.

Going with my original plan, did the surgery, the patient survived, and now has a fully intact memory!

Here's what was done:

- Manually recover/delete a file that was in the way of new $MftMirr & $UpCase copies

- Copy the backup boot sector up-1985 sectors & set its "hidden sectors" to 6081

- Copy $MftMirr & $UpCase up-1985 sectors

- Copy \System Volume Information\tracking.log up-1985 sectors

- Save-out the first 5 GB of the partition starting at boot sector LBA 4096

- Put that chunk back starting at LBA 6081

- Set the boot sector "hidden sectors" to 6081

- Change its MBR start sector to 6081

- Re-mount the drive/partition

The new (old) alignment now puts a file header at the start cluster of each file, which is so much more convenient, I must say. And the fragments don't contain parts of other files, which helps.

That 5 GB chunk procedure was a time-saver, as it contains all the rest of the metafiles, folders, etc, and luckily no data files near it. It can be put back at 4096 with minimal fuss, but so far it looks good at 6081.

MyDefrag is handy for visually locating metafiles in relation to data when planning what to copy and detecting possible overwrite conflicts, keeping in mind that it reports only what the file system has mapped, and that the data files are not inside those boundaries if they're not aligned. But it gives a good overall picture of what's where, with zoom capability.

Even if every metafile and log doesn't have to be moved, it's best to keep everything while we shift sector-alignment of the storage area. Since I don't know what can safely be left out, and I don't want the OS to re-create files or write anywhere that isn't already allocated to the file system, the partition should mount as if we had done nothing to it.

Once the files are copied off, the drive will be zeroed and re-partitioned with proper alignment under Win7. Technically, the fixed partition is functional as-is, but the drive controller has to do more work serving-up clusters that span the Advanced Format 4k sectors it uses natively. Solve one problem, create another...

[jaclaz]

Happy you made it.

jaclaz