porear

Hey guys! Just popped back over to the old thread today and found this new sub forum. Thanks very much for the mention jaclaz, you deserve much kudos for your insight and suggestions to point people in the right directions of attack. I have missed playing with this issue, but life continues to get more unpredictable... we had a baby, moved, and now are expecting again, so are looking to move once again. Not much time to tinker! Thanks to all those who started this, and have carried it on to what looks like an excellently successful fruition.

Hey guys! Sorry for a slightly OT post, just wanted to say hi and I am excited by your success. I have not been on here in a while, the new little man in our house has obviously changed our lives quite a bit. I am looking forward to finding some time to try out the ilko_t method, thanks very much to you and cdob, and of course jaclaz, who has been an excellent mentor in this whole process.

Happy New Year! Just dropping a quick note to say I'm still here and still have not abandoned this, but my time will be spotty for a bit. The wife is nesting, so I have quite a few home projects in progress, and I've also changed job assignments at work. Will contribute again as soon as possible.

I have been out of town and this "hobby" has been overtaken by enjoying time with family However, certainly not abandoned, will be back in January as well. Your findings are consistent with the Microsoft FAT spec http://www.microsoft.com/whdc/system/platf...are/fatgen.mspx FAT12 and FAT16 have a fixed root directory size, while FAT32 does not. I had assumed that this root directory limit would be enforced by the OS, and you would not be allowed to place more than the maximum entries in the root on FAT12 and FAT16. Maybe that is not the case. The answer is probably in the spec, which I'll look over more when I can. In the mean time, have a wonderful Christmas and Happy New Year!!

Sorry I've been unavailable for a few days. Thanks for the links jaclaz, I will give the tools a shot. I'll try to make sure they have the basic functionality needed, as well as try to test their relative performance (speed).

What version of dsfok do you have? I thought it was possible to start from an offset - I am using #4b. From the readme I've been researching to learn more about the FAT and directory structure on disk. Here are some good references. You may have found the same or very similar. http://home.teleport.com/~brainy/fat16.htm http://www.mcmillan.cx/fat.html From what I gather, there is no central master table for entries other than the root. The rest are spread out over the disk. The FAT has a single entry for every cluster in the data section on the disk. The entry in the FAT for each cluster indicates whether the cluster has data, is unused, or is the last cluster for a file. Directories are just a special file on disk with the table structure discussed in the second link above. An entry in the table points to the first block of data in the directory. From the references, as I (think I) understand, FAT entries in these ranges mean the following 0000h available cluster 0002h-FFEFh next cluster (or first?) in file FFF0h-FFF6h reserved cluster FFF7h bad cluster FFF8h-FFFF last cluster in file If this is correct, we would need to parse the FAT, and look for all entries in the 0002h-FFEFh range that follow an entry that is NOT in the 0002h-FFEFh range. These entries will point to the first cluster of each file. The first byte in these clusters should be the first letter of each file name that will need to be backed up and restored. (This needs to be verified) I also believe that for the situation to be this simple, we are assuming that files have no fragmentation and are completely contiguous. If not, we would have to follow the trail of the "next cluster" entries and where they point for the next piece of a file until we encounter a "last cluster" entry. I hope that helps and perhaps provided something you didn't already know. I also hope it is all correct! Idea: Maybe comparing before/after FATs will help identify pointers to the files that have been deleted, so that we only back up those that are necessary instead of all. Update: Here is the FAT spec from Microsoft. Have not looked at it in depth yet but looks like it should answer all our questions. The pages are titled FAT32 but cover FAT12 and FAT16 as well. http://www.microsoft.com/whdc/system/platf...are/fatgen.mspx

Wow that's fast, worked great. Output attached in two parts, the file was over the posting limit. Also, just out of curiousity, I used dsfo to make an image of the first 700MB of the stick to my hard drive. Restoring the image takes only a few minutes (less than 5) but this method requires an additional 700M of storage. I am sure it would have taken considerably longer had the backup image also been on the stick, and of course twice as much space on the stick would be used. Not the preferable way to go.

Progress. uuuuuuuggggggh I was using the wrong version of findpart - I had the DOS version. I've been able to run the batch, the first part for finding the FATs and ROOT seems to work fine. The directory entries part of the code did not work for me, I have not began to troubleshoot yet. My output is attached. Thanks!

Hello, ilko_t. Welcome and thanks for the efforts. Sorry I've had a short lapse on this, life's been busy - we're expecting a baby! Thanks for the code jaclaz. My batch script skills are lacking what is required for this task. I had been considering throwing together some pseudo-code for what was needed, but looks like you have it covered. The basic idea is to traverse the FAT and record the on-disk address pointed to of the first byte of each file entry (the first character of each file name). I tried the batch, but was unable to get it to work for me. I know I am most probably not using it properly I do have FindPart in the same directory as savesect.cmd. I ran the batch from a command line in Windows. I tried both Physical disk 1 (my hard disk) and Physical disk 2 (USB stick), with output redirected to 1.txt and 2.txt respectively. These files are attached, as are the screen outputs for each run in screen1.txt and screen2.txt. I tried also booting to DOS on the stick and running the batch there. I'm using DOS 7.1 from Win98, and COMMAND.COM does not recognize .cmd files. I changed it to a .bat, but it still won't run. It skips accepting input when asking for "YES" to confirm reading the boot disk, and immediately displays "LABEL NOT FOUND". I am assuming this is from a GOTO LABEL in the batch script. I've played with imaging the USB drive, then using gsar to search the image for the file names to provide the offset addresses. It works, but it finds 3 occurrences of each so we'd have to determine which is the one we want. It takes about 30 secs per file, which is actually pretty fast, but will take over a day for all 3000 files. This only has to be done once, but it's still extremely inefficient. A batch to process a list of all the file names would be needed too. Since the files are on disk in alphabetical order, the search could be shortened by starting subsequent searches at the sector where the last match occurred. Unfortuantely, gsar does not accept a file offset from which to begin a search. I am sure other utilities could be found that would. screen1.txt screen2.txt 1.txt 2.txt

I had seen (and even downloaded) RFD but passed it over. Maybe I need to look at it more carefully. It looks like it isn't free. EDIT The 1.4 version is free. You are right, we could regenerate from the XP installation back to the disk. Another option might be to simply have a second copy of all the deleted files on the disk and copy them back over each time, but I was hoping to avoid fragmentation. The deleted files total about 144 MB, which still take some time to copy on a USB, but it isn't unbearable. So far the closest I have found to what I have been looking for is NT_SS mentioned previously My best hits have come when searching for "physical disk search" and "hard disk forensics" This search for disk offsets only has to occur once, so (as you also have ascertained) we still have flexibility for the environment the search runs in. Even a GUI app that spits out a text report would do the trick... UPDATE: I took your suggestion of chainloading IO.SYS on the stick in GRUB. I was able to run Scaven! Not sure yet how fast it is. The main limitation with this approach is that the input file of search strings can only be 512 bytes - and our list is 40K. It could be split into 70 separate files and run by batch, but this seems a bit impractical.

Maybe not for you! hehe Very well done, sir. Works perfectly. I made a couple of very minor tweaks. IF NOT %counter%==8 set filename=%filename%#&goto :Loop1 I believe the #s actually need to be spaces (0x20). This next one threw me for a while. It was very strange, but the batch file made it through most of the directory then crashed. Turns out it didn't like the file "notepad.ch_" because when it parsed the name, it was literally translating the first three letters "not" as a NOT. I used some quotes and its works great. IF NOT "%filelen%"=="%filename%" SET length=%counter% &goto :loop1 I have been looking at Tiny Hexer scripting capabilities, and unless I am mistaken, "search" does not seem to be a scriptable function It is available in WinHex scripting capabilities, but not in the free version. We really just need a way now to obtain the hex offset of all of the file name locations. It still seems like with that additional piece of data this would be a suitable task for a batch that employs dsfi. We would not even need to rewrite the entire file name, only go to the offset where each name is located and rewrite the first 'å' byte. I found a tool that sounded suited for this at http://pjwalczak.com/scaven/index.php but alas it wouldn't see my USB disk, and the input file is limited to 512 bytes (our list is 26K). I've written the author but wouldn't consider this a very probable solution. This tool, NT_SS is even more perfect, but carries a $50 tag. Its functionality is exactly what I am seeking. http://www.dmares.com/maresware/html/nt_ss.htm I am attaching replace.txt. It has the deleted name on disk, the first character that was replaced for each name, the original name on disk, and the original file names. replace.txt

That sounds way too easy - now where is the fun in that? Just kidding. The problem I have had with that type of approach has always been that once the text mode setup has finished loading (the part that is on the floppies) that I cannot get the installation to continue since it asks for the CD. This occurs even if I have the CD tag files on whatever medium I am trying to install from. I am sure this could be solved, I just have not yet been able to do it. Regarding the Tiny Hexer approach, I think it is possible, but it will require another step: the file names on the disk are in 8.3 format, so to search for them, our match list will first have to be modified. Looking at the installation files, I do not believe any have names longer than 8 characters, but some have less. On disk, these shorter names are padded with spaces (0x20) at the end, e.g. boot.ini becomes BOOT INI (four spaces between boot and ini) Unfortunately some of the shorter deleted files have the same name with different extensions, so we cannot just search for åclui, we must search for åCLUI CH_ (three spaces in between for all of these) åCLUI DL_ åCLUI HL_ Simple enough, just another step.

Possible idea: Tiny Hexer at http://www.mirkes.de/en/home.php It's free, it's scriptable. Maybe there is a (scriptable) way to feed it the text file listing of file names, search for the location of each of the file names on the disk, and record these locations in the text file with the names. Then... create a batch or script file to use dsfi to restore the first character of each file name after deletion based on the text file created above. What do you think?

hehe and I as well. As a tip, I have found that the term "file system snapshot" is turning up some interesting results, but have not gotten in too deep yet. This might not actually be that hard to do. I know the 3,025 files in question that are deleted and could make the database easily. This database would contain only the filenames and not their locations, so a complete search would have to happen for each file. My only concern would be that if gsar searches the entire disk each time to make each replacement, this may be impractical for the time required. A more targeted approach that would use the FAT information to go straight to the file would use a fraction of this time.

uuuuuuuughhhhhh too much food. Hello, Puggsley and LLXX. I've been trying to determine what happens during a file deletion. Not only is the FAT altered, but one byte per file (I am assuming the first) is overwritten as well. It appears that this is the first character of the file name that must be manually supplied for most undelete methods. Experiment: I imaged through the end of my FAT tables (img1), then imaged the whole stick (img2). I deleted 3 files, restored the FAT from its image (img1), then re-imaged the entire stick again (img3). Comparing the two whole-disk images (img2 and img3) resulted in 3 differences found, in attached REPORT.TXT. The 42, 47, and 47 that were overwritten should correspond respectively to b, g, and g in the filenames bootlace.com, grldr.mbr, and grub.exe. So, for successful automatic undelete, the first character of all file names must be saved prior to deletion and then restored. I am assuming this is what "delete tracking" in the old DOS undelete did. http://free-backup.info/file-recovery-usin...s-undelete.html This would be perfect, unfortunately it requires a DOS 5.22 to DOS 6.0 environment to run. I've been looking but have not come across a similar tool that works in our XP Setup environment. From http://www.computerhope.com/undelete.htm I hope we're not digressing too much here to chase this as a solution to the file move issue, especially since this was already "Plan B" Although my talents are limited in this area, I might look at what it would take to try to write something that approximates the undelete "tracker". I ahd thought that although it complicates things, maybe we could alternatively boot to DOS, XP Setup, then DOS again and use the existing undelete with tracking. However, the tracking has to be running when a file is deleted for it to be added to teh tracking file. Report.txt

UPDATED See flagged note below I think my method of experimenting is flawed. I am using the disk offset of 282,112 (also tried 298,496) and addressing the USB with the scheme \\.\PHYSICALDRIVE1 I am doing all of this from within Windows XP (which I think is a problem) Save through FAT2 with dsfo \\.\PHYSICALDRIVE1 0 282112 fatsav1.dsk Delete several files from the stick Restore back to the stick with dsf1 \\.\PHYSICALDRIVE1 0 282112 fatsav1.dsk Save another file through FAT2 with dsfo \\.\PHYSICALDRIVE1 0 282112 fatsav2.dsk Compare the two saved files fatsav1.dsk and fatsav2.dsk with WinHex The files are identical, so the restore succeeds. However, the deleted files do not reappear in Windows Explorer. If then I do a View->Refresh of the USB drive in Windows Explorer, then save another snapshot with dsfo \\.\PHYSICALDRIVE1 0 282112 fatsav3.dsk Differences are found between this and the first two snapshots! I've attached REPORT.TXT generated by WinHex that shows the differences. It seems as if Windows XP has the first few sectors of the USB disk cached, and assumes that direct disk writes that bypass XP will not take place. The refresh action seems to write the XP cached data back to the disk. Is this a feasible theory? Update: I have tested this theory by removing the USB stick after restoring the FAT, then plugging it back in. Sure enough, this does seem to destroy any XP cached FATs, and the deleted files and folders reappear. HOWEVER... Only files and folders in the root are undeleted. Folders that originally held files that are undeleted to the root are now all empty. So, the root directory is being restored, but not the FATs If this is true, will the same problem arise in the setup/install environment? I have not tried it yet since my experiments have failed so far. I also am searching for other possible existing utilities for backup/restore of the FAT, or an undelete that will work in the install environment. I'll write back when I have anything new, but may not be on as much for the next few days, its holiday time here. I will also look at the link to the batch file you posted. Thanks!! BTW, I've been making it a point to honor the provisions of your CareWare license today, my friend. Report.txt

That should be correct. Thanks for the info and references, I'll look at them. This project is opening up quite a few new things to me. Beeblebrox reports C/H/S for the USB is 246/255/63. NumSectors in the partition table is 3963841. Using WinHex (and Beeblebrox to double check) to observe the bytes mentioned, 0x0E÷0x0F Reserved Sectors data is 04 00 0x11÷0x12 Root Entries data is 00 02 0x16÷0x17 Sectors per FAT data is F2 00 Choosing to jump to FAT1 in Winhex points to offset 0x800, and FAT2 is at offset 0x1EC00. From the data, this should be my drive: Offset 0x0 is Boot sector Offset 0x800 is FAT1 Offset 0x1EC00 is FAT2 Offset 0x3D000 is Root Offset 0x41000 is First data sector 0x1EC00 - 0x800 = 123,904. This length agrees with the calculation 242 sectors/FAT * 512 bytes/sector = 123,904 bytes/FAT. As always, thanks so much for the continued assistance.

hehe very nicely done. This thread was about due for a dose of OT Playing with dsfo now and trying to determine the length of the FAT, am thinking I read somewhere the boot record including FAT is 63 sectors, so 63 * 512 = 32256. Just how erroneous is that?

You are correct, its acting just like a move. I thought this might be the case and I had tried poking around in setupdd.sys with a hex editor but found nothing I could recognize to alter. I think that is an EXCELLENT idea. I had the blinders on and thought the only way to recover would be to recopy all the files again, I didn't consider it might only need an undelete. A candidate for initiating this might be the cmd switch when winnt32.exe is ran, apparently it somehow codes custom commands to be run later. I'll have to experiment unless someone is familiar with this switch, because the two references I have found about it do not seem to me to agree about when it happens: from http://winsupersite.com/showcase/win2k_cmdline_setup.asp and from http://www.quepublishing.com/articles/arti...qNum=6&rl=1

I wanted to submit a quick clarification that may cover some misunderstandings about the problem that is occurring. When text mode setup is ran from the stick, all the usual steps are encountered (pick a partition, format the partition or leave alone, check for appropriate disk space, etc). Then file copying begins. The files we are trying to save are each deleted one by one as they are copied, not as a batch at the end. If I let a few files get copied and interrupt the install, those files are already gone from the stick. This is a separate and more immediate issue than the deletion of the temp install directroies at the end of GUI setup. Hope this helps some... thanks!

I've not used EWF before. We're booting the FAT16 stick using NTLDR (and GRUB) into the setup environment using SETUPLDR.BIN. Could EWF be used here, or does it require you are already running Windows? Looks like its a RAM overlay that intercepts writes. Would you have to somehow feed it specific file names to watch for?

Hello, cluberti. Thanks for the response. The reason why we're trying to stop this deletion is due to an ongoing project over in thread http://www.msfn.org/board/index.php?showtopic=61384 attempting to install XP from (not to) a USB stick. Some of the individual files we're trying to keep are immediately deleted during the text mode setup as they are copied. At the end, the temp directories are deleted. Since our USB install method is based upon jump starting the setup process with the $WIN_NT$ directories placed upon the stick, unless we can stop file and folder deletion, we have only a one-shot solution.

Those are all good ideas, thanks so much for putting so much effort into researching this. Its definitely out of my experience level. I've posted this particular question back at the top (Windows XP) level of the forum so maybe the added visibility will atttract the attention of a setup expert. In our case for these specific files, it is happening before this stage. That entry is in the final GUI mode stage, whereas our files are deleted as they are copied while still in text mode in the very early stages. Its actually here, which is step 10 of the Initial Stage: However, you are right, once we get there that we will be looking for a way to stop deletion of the entire $WIN_NT$.~BT and $WIN_NT$.~LS temporary directories during final install cleanup. I believe this is the step 10 you mention.

In the initial stages of XP setup, some of the installation files are copied from the installation media to the hard drive where XP will reside. One directory of such files that is created is $WIN_NT$.~LS . During the next text mode portion of XP setup, some of the files copied from the temp directory $WIN_NT$.~LS are deleted as soon as they are copied into the final installation directory. Is there a way to stop this deletion from happening? Is there also a way to stop directories $WIN_NT$.~LS and $WIN_NT$.~BT from being deleted as setup completes? I've looked at TXTSETUP.SIF but cannot see anything there that distinguishes the individual files being deleted from those that are not deleted. I have not been able to find a definitive guide to explaining all the switches used in TXTSETUP.SIF. Such a reference would be greatly appreciated! Thanks

That worked. The new menu.lst is color black/cyan yellow/cyan timeout 10 default /default title Phase 1 WinXP Text Mode Setup from USB rootnoverify (hd0,0) chainloader (hd0,0)/setupldr.bin savedefault 1 boot title Phase 2 WinXP GUI Mode Setup from Hard Drive map (hd1) (hd0) rootnoverify (hd0,0) chainloader (hd0,0)/ntldr savedefault 0 boot With regards to the renamed GRLDR, this note in the GRUB4DOS readme.txt might apply?: I am still working the other file deletion issue, I've printed out TXTSETUP.SIF and the deleted files list. So far in doing a visual comparison, nothing jumps out.