Jump to content

How can I write this very specific batch file


Recommended Posts

Hello guys I have a question regarding a batch file. It's pretty specific so here goes.

We had some data corruption in our company recently. Now I restored the data from our backups however not ALL the data is corrupted. It seems that the data after a certain date is corrupted only.

The problem is there are 500k files.

What I would like to know is if someone can maybe write up a simple dos batch file that would do the following

Copy files modified only after Oct 2 2013 and retain the permissions.

Is that possible?

Link to comment
Share on other sites


http://superuser.com/questions/308277/is-there-anyway-to-xcopy-just-changed-or-new-files

xcopy /d parameter should do the first part of trick, you are on your own to find the permissions aspect. I'll check back later

http://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/xcopy.mspx?mfr=true

/d[:mm-dd-yyyy] : Copies source files changed on or after the specified date only. If you do not include a mm-dd-yyyy value, xcopy copies all Source files that are newer than existing Destination files. This command-line option allows you to update files that have changed.
Edited by MrJinje
Link to comment
Share on other sites

Robocopy is the tool intended to be used to keep all permissions:

http://ss64.com/nt/robocopy.html

Since parameters are complex and not very intuitive, I would recommend you to do a few experiments before issuing the command on the "bulk" of files.

Some help may come by the use of one of the GUI's for it (a few more are listed at the bottom of the page for this one):

http://betterrobocopygui.codeplex.com/

Since /COPY:DATS /COPY:DAT is "default", it is very possible that a simple command like:

http://superuser.com/questions/562375/how-to-copy-only-new-files-and-only-those-that-are-modified-after-a-certain-date

Robocopy c:\ e:\ *.* /MAXAGE:20130301 /XO /E

would do, as in the above link, try with the /L switch to verify if the result is the one expected.

BUT THE ABOVE WILL NOT COPY THE SECURITY AND OWNER PERMISSIONS, so you will need to add the /COPY:DATSOU or the equivalent /COPYALL switch.

Also, you should check if the actual permissions/security info of the backed up files is valid (this may depend on HOW exactly the backup was made and on which filesystem).

jaclaz

P.S.: Edited to correct wrong info, see below post by Yzöwl

Edited by jaclaz
Link to comment
Share on other sites

As jaclaz suggests above, robocopy is the current built in tool for the job.

You will need the /MAXAGE switch using 3rd October 2013, (/maxage:20131003), for the permissions you may find the /COPYALL switch best, since the default for /COPY is DAT, (not DATS - jaclaz :whistle: ), meaning that it will ignore the NTFS ACL as well as Owner and Auditing information.

Link to comment
Share on other sites

As jaclaz suggests above, robocopy is the current built in tool for the job.

You will need the /MAXAGE switch using 3rd October 2013, (/maxage:20131003), for the permissions you may find the /COPYALL switch best, since the default for /COPY is DAT, (not DATS - jaclaz :whistle: ), meaning that it will ignore the NTFS ACL as well as Owner and Auditing information.

My bad :blushing:.

So it is needed to add the /SEC parameter or explicit /COPY:DATSOU (or as suggested use the /COPYALL).

Editing previous post to avoid confusion....

Anyway testing with a subset of files is recommended.

jaclaz

Link to comment
Share on other sites

Hello again, so, I tried the batch file here is my command

Robocopy c:\Path to my files x:\path to destination *.* /MAXAGE:20131002 /XO /E /V

So as far as I can tell if I understand this right the MAXAGE of this file would be Oct 02 2013. I added the /V for verbose I wanted to see what it did it real time.

So I ran the batch file but it did not complete as expected. My test was I opened a file from my backup set that was good. This file was dating November 10 2012. After running the batch file I tried to open the same file and it is still corrupted. So I am led to believe that the batch file is not overwriting files that were modified AFTER the date Oct 02 2013. The file on the production server that is still corrupted though still has the last modified date as November 10, 2012.

Perhaps I am understanding the batch file incorrectly but does it not say "if the file is older than 20131002 do not touch it. If it is overwrite it" ??

Would there be another way of going about this to overwrite the corrupted files with the good files without doing it one at a time?

Thanks!

Link to comment
Share on other sites

The /FFT switch often helps when using modified dates/times and if you want to see the results without actually copying use the /L switch.

What results do you get using this?

ROBOCOPY "C:\Path to my files" "X:\Path to destination" /FFT /L /S /COPY:DATS /DCOPY:T /XO /MAXAGE:20131002
Link to comment
Share on other sites

You may not have been specific enough in Post #1, and/or I am not understanding this.
To me, your Post #1 scenario reads like this;
Backup (source good files)
Past 2 Oct 2013 Near Present
. ^
---------------------------|-----------------------
. | files modified after ~2 Oct 2013=definately not corrupted.

***

Production (destination) server
Past 2 Oct 2013 Present
. ^
---------------------------|---------------------------
. | files modified after ~2 Oct 2013=maybe corrupted!


If above is correct representation, then wouldn't you want to copy from Backup (source) TO
Production (offline destination) server, only those files (in the Backup) that have a
modification date "after Oct 2 2013 and retain the permissions", overwriting in the Production
(offline destination) server as appropriate.
-
You mention (after running the Robocopy batch), a file bearing date 10 Nov 2012. (I assume you
are referring to its modification date!) Huh? Why would any file bearing that modification date
have any corruption in either source or destination - it was never a 'file modified aft 2 Oct 2013'.
Please clarify.

Edited by buyerninety
Link to comment
Share on other sites

Ok I'll try to draw this out

BACKUP FILE SET

Backup (source good files) is my backup from OCT 2 2013
Past 2 Oct 2013 Present
.
---------------------------|-----------------------
. Good Files

DESTINATION SERVER (IN PRODUCTION)

Past 2 Oct 2013 Present
. ^
---------------------------|---------------------------
. | Files with date modified AFTER October 02 2013 are believe to be corrupted. This is not unique to only these files. Files with dates modified BEFORE this date may also be corrupted. We do not know.

My current batch file reads the following: Robocopy c:\Path to my files x:\path to destination *.* /MAXAGE:20131002 /XO /E /V

It reads that it will overwrite files that have a date modified AFTER October 02 2013 (if I wrote it write which I believe I did)

My issue : In running my batch file it did not overwrite many files. Once completed, I verified if some files that were corrupted before the restore were still corrupted and they were. Upon closer inspection of the file it came to my attention that the date modified was November 10 2012 (before the date my batch file has). Naturally, my batch file would not have copied over this file.

My question is the following: Is there a way for me to write a file that would verify the integrity of the file (on the destination ((production server)) ). If the file on the destination is corrupted then can the file from my backup set would over write it?

Link to comment
Share on other sites

Thankyou, I'm getting a handle on it now. Unfortunately I don't have an answer to Post #9's last paragraph ATM.
However, I can reason that the current Robocopy command line seems deficient.
Robocopy c:\Path to my files x:\path to destination *.* /MAXAGE:20131002 /XO /E /V
Here is what I believe it is commanding;
1. RoboCopy Source to Destination (All Files - but modified by below)
2. Exclude source Files w Mod date Before 2 Oct 2013 aka Don't Overwrite destination Files with Mod date After 2 Oct 2013
3. Excluding older files aka Don't Overwrite Files (in Destination) having Same or Newer Mod DateStamp than that in Source
4. AND Copy SubDirectories, even if those SubDirectories do not contain the specified file(s)
5. AND Report Verbosely, including skipped files.

Notes;
1.= Actually; copy Source Dir (with files) to Destination Dir. (The All Files is implicit [default], so " *.* " could be deleted;
other examples of command lines seen elsewhere seem happy to leave *.* out .)
2.= I'm willing to be educated here; does a later option override an earlier one? because 3. seems to me to override 2. !?,
or adversely modify it, or potential to adversely interact with it.
3.= I'll give refs if you want.
4.= Self explanatory (default behaviour without /e or /s is to only copy SubDirs containing specified files)
5.= Self explanatory, (you know better than me if this is directed to screen or printer.!)
Others should be putting better suggestions for the Command Line than me, otherwise I'll have to wade in. (Yzöwl excepted...)

Edited by buyerninety
Link to comment
Share on other sites

YzOwl I accidentally skipped over your post. I didn't read it. I will try this right now.

ROBOCOPY "C:\Path to my files" "X:\Path to destination" /FFT /L /S /COPY:DATS /DCOPY:T /XO /MAXAGE:20131002

I will post the results of what I get.

buyerninety

I am led to believe the command is saying the following (however I could be wrong in the interpreatation)

Copy all files from source directory to destination directory / exclude files older than 2013-10-02 / copy empty subdirectories / report verbosely

I will try Yzowls command now and see what results it gives and I will post my results.

I have ran the test.

These are the results it yeilded

Total Copied Skipped Mismatch FAILED Extras

Dirs : 5726 5725 1 0 0 0
Files : 93203 4010 89193 0 0 0
Bytes : 18.132 g 1.337 g 16.794 g 0 0 0
Times : 0:01:01 0:00:00 0:00:00 0:01:01

If I am reading this right there are 5726 directories and it would have had something to do in 5725 directories

There are 93203 files and it would have copied 4010 files

There is a total of 18.132 GB in these files and the command would have copied 1.337 GB worth of files over.

Am I understanding this correctly?

Ended : Fri Oct 25 09:52:31 2013

Edited by anthonyaudi
Link to comment
Share on other sites

Okay, before the tests go any further, I think we need to be sure what your intentions are. If you are overwriting with the backup set taken on 2nd October then all of these switches may be pointless. It just depends upon when you believe the files became corrupted.

Robocopy by default will overwrite all existing files in the destination if the source file size or time stamps are different. It doesn't care whether the destination file is older or newer, larger or smaller, if it is different then it will be overwritten by the source file.

You should theoretically get away with this:

ROBOCOPY "C:\Path to my files" "X:\Path to destination" /L /S /COPY:DATS /XL /W:5
It should replace every file on the production server which has changed since the backup set was created with that backed up file. I have included the /XL switch to prevent files and directories which no longer exist on the production server from being copied back to it, (they may have been deleted for valid reasons).

Remember the /L switch will need to be removed when you're happy with the listed output.

Link to comment
Share on other sites

Hello, so I got that batch file running everything seems to be good I verified a few files that were known to be corrupted on the production server from the restore everything is fine.

Now my next and final question would be the following: Is robocopy capable of putting the new files back into the right directories by simply overwriting the broken files.

What I mean is this:

Lets say I have on the production server c:\test\abc <--abc has no corrupted files HOWEVER c:\test has corrupted files

Is robocopy able to replace the files in c:\test without erasing c:\test\abc

(I'll try to use this is more simple terms)

If I have a folder called C:\Test and in c:\test I have a few subdirectories called 1/2/3/4 and so on.

And on another partition I have another directory called Test and in that directory I don't have any subdirectories.

If I did a COPY of the directory Test with no subdirectories and PASTED it onto the partition with C:\Test\ with the subdirectories it will overwrite that entire folder and erase my subdirectories.

Can robocopy know that I just want to replace the files in the destination with the files from the source without erasing everything else in the directory?

I'm pretty sure Robocopy can do this I was looking at the switches I would think its either the /XX or /IS switch.

But then again I am not 100% sure that's why I'm asking the experts!

Thanks a bunch guys.

Link to comment
Share on other sites

RoboCopy will not delete anything from your destination unless you use one of the following switches, /PURGE, /MIR, /MOV or /MOVE.

If you do not want to copy any directories, only files, then do not use the /S or /E switches.

If you wanted to action only the top level of files, i.e. 0 LEVels of directory, then you could simply use /LEV:0

If you wanted to eXclude specific Directories from being actioned, then use /XD followed by the directory name(s).

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...