Jump to content

Hard Disk Cloning/Imaging from inside Windows


JorgeA

Recommended Posts

Well, I thought I finally had a handle on this business of "disk cloning" and "disk imaging," when I saw these two posts by @jaclaz and @submix8c in another thread:

You NEVER, and I mean NEVER image a disk/volume that is in use (unless you use Shadow Copy services).
Hmmm... you can't clone a "running" XP. No way. HIVES are actively being accessed.

So... there is a lot of well-known and -respected third-party software out there that promises to do "bare metal" or "complete system image" backups of your boot drive, such that in case of a HDD failure you can re-image the backup (typically from a USB-connected external HDD) to a new internal primary drive. In the cases I'm thinking of, these software packages create their backup images from within Windows.

But if I understand what jaclaz and submix8c are saying, then it is not correct to say that these software packages will restore your system to working condition in case of a catastrophic HDD failure. Are these vendors deceiving buyers? What's the deal? And if jaclaz and submix8c are right, then what is the proper, step-by-step procedure for creating an all-encompassing, thorough, full, complete, comprehensive, total, all-inclusive, bootable and working OS drive backup that you can use to re-create your system in such catastrophic failures?

Or is the issue, perhaps, that they're talking about XP and older versions of Windows, whereas the vendors' promise does hold true for Vista and Windows 7?

I've read about this topic in Wikipedia, and I'm no closer to understanding jaclaz and submix8c's point than I was before I started. Not terribly helpful encyclopedia articles, at least when it comes to the point our MSFNers are making.

Enlightenment is welcome. :)

--JorgeA

Edited by JorgeA
Link to comment
Share on other sites


Most OS now have shadow copy service. Most cloning software uses that. That said, Jaclaz is factually correct, but tends to over-dramatize, also your taking it out of context. Mostly that was being said to a single user who wanted to do something that just couldn't be done.

I believe the reason was because a user had wanted to "robocopy" clone his image without using a 3rd party cloning software. and we all know one does not simply file-copy clone Windows XP while the OS is running. It would have worked from WinPE, but not from inside XP. OP couldn't run a live CD because he had to remove his CD drive to caddy his SATA. He had a craptop, not a laptop. I think he ended up removing the HDD and cloning it elsewhere.

The user in that post could have simply installed acronis, macrium reflect or what have you, but he made it clear he did not want to install 3rd party cloning software. These companies are not scamming anyone, most of their products work just fine while the OS is running. Make sense ?

Link to comment
Share on other sites

He had a craptop, not a laptop.

It was the HDD not the laptop. Get your facts straight before opening your mouth.

All my laptops run the latest hardware. I have replace that drive with another 120GB SATA, and will probably buy another this weekend just to have it on deck. I don't buy cheap things. I do top of the line buddy :w00t:

Link to comment
Share on other sites

I guess we are always around the same terminology issues.

  1. Cloning.
  2. Imaging.
  3. Backing up.

Let's see if I can explain the basics.

A clone is an EXACT, byte by byte, copy of a disk (normally whole disk).

If you prefer there is NO way to distinguish the copy from the original. (set apart the actual hardware disk serial/make/model/capacity).

An Image (a "real" or "forensic sound" or "dd-like") is the same as above, stored *somewhere* and that once deployed to a disk the target will become a clone (again No way to distinguish it form the original).

Is such a "forensic sound" image (or the "clone" that you can create directly or through dploying this "dd-like" image) needed for *everything*? :unsure:

Certainly not :no: .

It is NEEDED; VITAL for:

  • forensics
  • data recovery
  • whatever that requires an EXACT copy

For other uses (such as "bare metal" recovery) more than a few things can be removed from it, an incomplete list being:

  • unused sectors (no matter if actually 00's or simply unindexed space)
  • slack space (see here for a NTFS oriented list) : http://www.forensicfocus.com/Forums/viewtopic/t=9374/
  • pagefile.sys (if any)
  • hyberfil.sys (if any)
  • temp folders contents and more generally caches, histories, cookies, etc.
  • (usually) hidden sectors
  • ....

So, if we put the whole disk at 100%, a clone or a forensic sound image will be 100%, while an image done for other scopes may be anything between a very low value (for an almost empty disk) up to 99.99% (but NEVER reach 100%), more likely it should be at something like max 70%.

Then there is another issue, the WHEN.

A clone or a forensic sound image are made rigorously OFFLINE at a given date/time.

Immediately after (and until you don't boot either the original or the copy/clone) they are IDENTICAL.

Things like last time accessed timestamps are preserved.

A more "normal" image CANNOT be made ONLINE IF NOT through using the Volume Shadow services (or equivalent third party approaches/methods).

What the Shadow copy does is to create a "temporary" unchanged version of files.

So what you copy is not actually what "is" on the system, but what "was" on the system.

Since it is called Volume Shadow, the object is the Volume (and NOT the disk), so a solution (BTW Third Party) such as DriveImageXML is NOT a disk imaging solution (because it is a drive or volume imaging one) and while "enough" for many purposes it is NOT enough for "bare metal" recovery and an image made with it needs to be "integrated with some other data/through other tools, see: http://www.911cd.net/forums//index.php?showtopic=22984

A backup is - generally speaking - meant for DATA (not for disks, nor for drives or volumes).

Just like the above described "simplified image" it needs to be integrated with some other data/through other tools (evidently more additional info is needed when compared with a Volume image).

Summing up:

  1. CLONE=deployed dd-like IMAGE
  2. dd-like image = 100% or WHOLE DISK (including EVERY sector of the disk and thus all Volumes/Drives and all info external to Volumes/Drives) ACCURATE
  3. Shadow Copy Image=70% (single volume disk) INACCURATE needs additional (very few) info and or tools for "bare metal" use (info external to Volume/Drive)
  4. backup =50% (say) VERY INACCURATE needs additional (not very few) info and/or tools for "bare metal" use (info external to Volume/Drive AND info about Volume/Drive)

This said, there all all shades of grey, just as an example this nice tool (severaly misnamed as XXCLONE) does NOT "clone" but provides an "intermediate" approach:

http://www.xxclone.com/

But the original question was "without third party tools".

Some further considerations are here (hardware based cloning):

jaclaz

Edited by jaclaz
Link to comment
Share on other sites

If you allow yourself to use freeware third party tools, then you can make a bootable copy of a running OS from xp onwards:

http://blogs.msdn.com/b/adioltean/archive/2006/09/18/761515.aspx

This doesn't handle junctions though, one day I'll try strarc instead of robocopy.

I have boxes that copy themselves to iscsi lvm partitions, then in a disaster I can boot the copies in kvm.

Link to comment
Share on other sites

Without installing third party tools. And I had a valid suggestion for that. :yes:

Which was actually the SAME one I had proposed LONG BEFORE you did, i.e. booting to "something else" :whistle:

Compare:

Please define "install".

Many of the solutions provided are about boot-CD's or bootable USB sticks or floppy.

You NEVER, and I mean NEVER image a disk/volume that is in use (unless you use Shadow Copy services).

You need to boot the machine to ANOTHER OS (that contains the needed tools), this can also be a "dual boot" with a "hidden, recovery like" partition on the main hard disk.

So, in any case you NEED a "third party tool", whether this needs to be "installed" or not, is another thing.

with:

Anyway you look at it, you need to be able to boot and clone from something else than your XP.

and I can assure you that the suggestion I made (BEFORE yours) is as valid and possibly much simpler, and there are possibly several more very good alternatives to both.

BUT, this thread is not the one where we discuss what PROBLEMCHYLD could or should do in order to solve his problem, this thread was started by JorgeA that asked a question about two statements (BTW perfectly valid) that submix8c and myself did in that specific thread and by taking it out of context managed to derive from them a doubt that needed to be cleared (the possibility to have a "good enough for bare metal restore from within an INSTALLED THIRD PARTY TOOL and a running XP).

Your nice suggestion about a not installed third party tool in order to solve PROBLEMCHYLD's issue has obviously no actual relevance on a question about an installed third party tool used in another context.

To try and clear the doubts:

  • "third party tools" (NOT INSTALLED) do exist that can do a "bare metal recovery" <- from a "non-live" XP system
  • "third party tools" (INSTALLED) do exist that can do a "bare metal recovery" <- from a "live" XP system
  • there are ways by combining "third party tools" (EITHER INSTALLED or NOT INSTALLED) with built-in tools like cdob :thumbup suggested on the referenced thread (NTBACKUP) <- from a "live" XP system

jaclaz

Link to comment
Share on other sites

I guess we are always around the same terminology issues.

  1. Cloning.
  2. Imaging.
  3. Backing up.

[...]

jaclaz,

Thanks for the info-filled reply!

My question was indeed somewhat different from PROBLEMCHYLD's, which is why I started a new thread rather than confuse things (further) over there. In my case I'm not averse to installing third-party software to do what I need done.

OK, so here's the scenario I have in mind. Suppose that my HDD fails tomorrow -- the electronics on the drive go kaput, the magnetic head scrapes against the surface with a chilling noise, and there is no way to revive the drive, much less to recover the stuff that's on it. What type of procedure and which software do I need to have completed and ready today, so that tomorrow when the HDD dies I can simply take it out, put a fresh HDD drive inside, and restore my entire system to the state it was in today so that I can pick up where I left off?

Note that I'm not looking for solutions that involve reinstalling Windows and reinstalling all my programs and settings. That is a major PITA that I would reserve as punishment for my enemies. I'm looking for something that will restore everything the way I have it today, without further manual tweaking. (OK, I'm willing to allow for restoring, let's say, word-processing and mail files from outside backup sources -- it's the OS and programs and drivers and settings that I do not want to have to recreate by hand.)

Given that goal -- will something like Acronis True Image (it came with my Seagate external HDD, branded as "BlackArmor Backup") accomplish what I'm looking for? Or if not, then what do I need to do?

BlackArmor Backup offers to do (among other and less extensive types of backups) "images" in either "compressed" or "uncompressed" form. I alternate each kind every other month, the idea being that I'll never be more than 30 days behind and I can bring my data files up to date from less-extensive data backups that I keep on a different external HDD.

Bottom line: what I ultimately want to know is, In case of a catastrophic HDD failure, will one of these types of BlackArmor Backup "images" serve my purpose? Based on what you said and what I've seen before, I think it will do the trick. But do I have that right? :unsure:

--JorgeA

Edited by JorgeA
Link to comment
Share on other sites

Given that goal -- will something like Acronis True Image (it came with my Seagate external HDD, branded as "BlackArmor Backup") accomplish what I'm looking for? Or if not, then what do I need to do?

BlackArmor Backup offers to do (among other and less extensive types of backups) "images" in either "compressed" or "uncompressed" form. I alternate each kind every other month, the idea being that I'll never be more than 30 days behind and I can bring my data files up to date from less-extensive data backups that I keep on a different external HDD.

Bottom line: what I ultimately want to know is, In case of a catastrophic HDD failure, will one of these types of BlackArmor Backup "images" serve my purpose? Based on what you said and what I've seen before, I think it will do the trick. But do I have that right? :unsure:

--JorgeA

Sure :), it is designed to do that, as long as you ALSO have a way (typically a CD/DVD and/or a USB device or second instance of OS on another disk) to actually initiate the recovery procedure.

I.e. the thingy is installed to "backup" into the "live" system but needs to boot (in order to "restore") from "external" media (as the new disk will obviously be "blank").

A good idea, as always when it comes to delicate things such as backing up and restoring, is:

  1. DO NOT trust the vendor/Author of the tool :no:
  2. DO NOT trust what a guy on a board says :ph34r:
  3. DO NOT trust yourself :w00t: (in the sense of not trusting the presumed familiarity you have with the tool, the way you understood it should be operated and/or the way it works or it is advertised as working)
  4. actually get a "new" disk and simulate the backup and recovery process BEFORE disaster happens :yes: and see if the result is what you expect

If you had actually READ the manual for that thingy:

http://www.seagate.com/staticfiles/blackarmor/userguides/BlackArmor_UG.pdf

You would have probably found out most of the concepts I briefly summed up.

The biggest "risk" with tools like that one (which can do several types of operations/different kinds of backups/images) is the actual way it is used, if you prefer there is a risk of a PEBCAK :rolleyes: .

AFAICT, it represents a "good" software, though I personally do not like that kind of software (like any other similar software that "hide" the complexity of operation behind a "few click away" APPARENT simplicity) and I prefer to use simpler (most people would call them "more rudimental") software/tools.

In a nutshell:

Can it do it? Yes.

Will it do it? Cannot say.

Will it do it when used by you? Cannot say even more....

jaclaz

Link to comment
Share on other sites

jaclaz,

I appreciate the information, however, the following line was totally uncalled for:

If you had actually READ the manual for that thingy:

http://www.seagate.com/staticfiles/blackarmor/userguides/BlackArmor_UG.pdf

You would have probably found out most of the concepts I briefly summed up.

OF COURSE I have RTFM -- more than once, even. However, I have found that the selfsame FM (as well as the "help" function inside the software) is p*ss-poor, leaving too many questions unanswered and ideas inadequately explained.

Now combine that poor experience with the manual with your statement in the other thread, to wit:

You NEVER, and I mean NEVER image a disk/volume that is in use (unless you use Shadow Copy services).

Therefore, since BlackArmor Backup makes its images/clones/backups/whatever WHILE THE DISK IS IN USE, it was not unreasonable for me to start wondering whether that software would do what I need to have done. Sorry, but computers are not my living -- I use computers to make a living; therefore, as Windows is not the main focus of my professional life, but only a derivative interest, I am not entirely familiar with the minutiae. Hence my presence on this Forum, one of whose purposes I've always understood it to be to ask questions and provide answers about MS (as in MSFN) products. Preferably, without having one's intelligence or diligence questioned.

The questions I had about this are now answered. Thank you for THAT part.

--JorgeA

Edited by JorgeA
Link to comment
Share on other sites

OF COURSE I have RTFM -- more than once, even. However, I have found that the selfsame FM (as well as the "help" function inside the software) is p*ss-poor, leaving too many questions unanswered and ideas inadequately explained.

Now combine that poor experience with the manual with your statement in the other thread, to wit:

You NEVER, and I mean NEVER image a disk/volume that is in use (unless you use Shadow Copy services).

Therefore, since BlackArmor Backup makes its images/clones/backups/whatever WHILE THE DISK IS IN USE, it was not unreasonable for me to start wondering whether that software would do what I need to have done. S

Well, this is flattering :blushing: you value a single sentence extrapolated from a post of mine in a thread where the issue is "not installed third party" tools more than a whole manual from guys that since several years (I am talking of Acronis, not of Seagate) are among the "top players" in data backup/disk imaging/cloning.

When you actually read the manual, you must have missed chapter 3.1:

Chapter 3. General information

3.1 The difference between file archives and disk/partition images

A backup archive is a file or a group of files (also called “backups” in this guide), that

contains a copy of selected file/folder data or a copy of all information stored on selected

Seagate BlackArmor NAS or Seagate\Maxtor DAS.

When you back up files and folders, only the data, along with the folder tree, is compressed

and stored.

Backing up disks and partitions is performed in a different way: Seagate BlackArmor Backup

stores a sector-by-sector snapshot of the disk, which includes the operating system, registry,

drivers, software applications and data files, as well as system areas hidden from the user.

This procedure is called “creating a disk image,” and the resulting backup archive is often

called a disk/partition image.

By default, Seagate BlackArmor Backup stores only those hard disk parts that contain data

(for supported file systems). Further, it does not back up swap file information (pagefile.sys

under Windows XP/Vista) and hiberfil.sys (a file that keeps RAM contents when the

computer goes into hibernation). This reduces image size and speeds up image creation and

restoration. However, you might use the Create an image using the sector-by-sector

approach option that lets you include all of the sectors of a hard disk in an image.

A partition image includes all files and folders. This includes all attributes (including hidden

and system files), boot record, and FAT (file allocation table); as well as files in the root

directory and the zero track of the hard disk with master boot record (MBR).

A disk image includes images of all disk partitions as well as the zero track with master boot

record (MBR).

which seems to me like very similar to what I summed up in post #5.

Therefore, since BlackArmor Backup makes its images/clones/backups/whatever WHILE THE DISK IS IN USE, it was not unreasonable for me to start wondering whether that software would do what I need to have done. Sorry, but computers are not my living -- I use computers to make a living; therefore, as Windows is not the main focus of my professional life, but only a derivative interest, I am not entirely familiar with the minutiae. Hence my presence on this Forum, one of whose purposes I've always understood it to be to ask questions and provide answers about MS (as in MSFN) products. Preferably, without having one's intelligence or diligence questioned.

Yep :yes: , but you cannot play dumb at will :no:.

The average Joe would take what the good Acronis or Seagate guys say on the matter as "the one and only thruth" and happily image/backup/whatever his disks, the sheer moment in which you decide to delve deeper, and doubt their word for it, you will need to get familiar with the minutiae :whistle:.

Red pills are better IMHO, but rabbit holes are deeeep :ph34r: :

jaclaz

Link to comment
Share on other sites

  • 2 weeks later...

Well, this is flattering :blushing: you value a single sentence extrapolated from a post of mine in a thread where the issue is "not installed third party" tools more than a whole manual from guys that since several years (I am talking of Acronis, not of Seagate) are among the "top players" in data backup/disk imaging/cloning.

When you actually read the manual, you must have missed chapter 3.1:

[etc.]

Well, what can I say -- I do trust your opinion better than the BlackArmor Backup manual, which has left me scratching my head numerous times. And I've also been around computers long enough to know that what the manual says, ain't always so.

It's been said that I took out of context your original statement that

You NEVER, and I mean NEVER image a disk/volume that is in use (unless you use Shadow Copy services).

...except that the statement was a.) not qualified by anything, and moreover b.) quite emphatic about its statement. When a person says "never" -- and twice -- I take it that they mean to say "never" (as in, under no circumstances). Can't get any clearer than that. How was I supposed to know you meant "never in these particular circumstances" -- and no, when it comes to software, I've been burned enough times to cure me of the habit of assuming.

Oh, and as far as the high quality of the work by those Acronis folks -- their software crashes (fails to make a backup) as often as not. And I have given up completely on their function for scheduled backups, which fails MORE often than not. Their software is fussy and its output unpredictable, therefore I cannot be certain as to its reliability. And their vaunted manual has been ZERO help in explaining the source/cause of the software's failures, ergo I put little stock in what the manual has to say. I am less than impressed.

The BlackArmor Backup software is so bad that I uninstalled it this month. (See why I'm so eager to inquire about the reliability of backup software?) In its place I installed the Seagate DiscWizard, which appears to be a less ambitious package. We shall see whether it actually works any more reliably.

With that as my background, when I read a trusted expert write that, "You NEVER, and I mean NEVER image a disk/volume that is in use (unless you use Shadow Copy services)," I hope you will come to understand why I posed my original question. I'm dubious about my software to begin with, and then I read a statement like yours that seems to fit the conditions under which it operates (= running while the disk is in use).

So excuse me for asking for help from someone I've come to trust, on a topic (1) that I don't know much about, (2) for which extensive reading has not clarified my doubts, and (3) in which my limited experience has taught me to question the trustworthiness of backup software.

The average Joe would take what the good Acronis or Seagate guys say on the matter as "the one and only thruth" and happily image/backup/whatever his disks, the sheer moment in which you decide to delve deeper, and doubt their word for it, you will need to get familiar with the minutiae

That's exactly the reason that I started this thread -- because, given my experiences with backup software, I wanted to delve deeper into the topic. As I see it, the "average Joe" would happily image/backup/whatever his disks, and then find out when his HDD crashes and he really needs the backup, that it failed due to the sort of obscure technical reason that you appeared to be alluding to. I simply wanted to learn more about this.

--JorgeA

P.S. Recommendations for reliable imaging software are welcome, from any source.

Edited by JorgeA
Link to comment
Share on other sites

Very good. :)

I am sorry about your misadventure with the Acronis/Seagate thingy :( , as said I don't particularly "like" Acronis software but AFAIK it normally works and I don't think that the good Seagate guys did anything to it (besides changing name/re-branding) in order to render it inoperable :w00t: .

Thus the issues you are experiencing are somewhat preoccupying. :ph34r:

It is clear that that software, in order to be able to work from the booted windows, uses either Shadow Volume services or a similar "Acronis made" approach.

It is possible that you have some other software interfering with it's operations.

You could try using a freebie like DriveImageXML (which as well normally uses Shadow Volume services) combined with *any* tool capable of backing up the MBR (or in case of a multisector MBR or to be anyway on the "safe side" the whole set of hidden sectors):

http://www.runtime.org/driveimage-xml.htm

You will need to make a BartPE (or similar) boot CD (or bootable USB stick) to be able to do a "bare metal" restore.

This way, you can determine whether the issue relies on Shadow Volume subsystem or it is limited to the Blackarmour thingy.

These could be related valuable sources of info:

http://www.911cd.net/forums//index.php?showtopic=22563

http://www.911cd.net/forums//index.php?showtopic=23680

http://www.911cd.net/forums//index.php?showtopic=22984

What I personally see as a good thing of the app is that it saves files that are (with some "tricks") "human readable", thus it is possible to recover manually the data in them/correct them, example:

http://www.911cd.net/forums//index.php?showtopic=23408

This said, I still prefer, since it is inherently safer (as it doesn't rely to Voume Shadow or similar mechanisms) to boot *something else* and use this "temporary" OS to backup/image.

Personally (and since the dawn of time) I tend to have two installs of the OS on two separate partitions, one "minimal" (nlite, etc.) and one the "real" one, used everyday.

You backup (actualy directly image since it is small) the "minimal" from the "real" (you need to do this only once).

Then you peridically boot to the "minimal" and from it backup or image the "real" (and the "data" volumes).

In the (fortunately rare, but unfortunately possible) case of a total hard disk crash, you restore the "minimal" (from a PE) and from it restore all the others.

BTW with this approach the "built-in" in 2K and XP NTBackup is not that bad, but which OS are you actually running?

The same approach can be made obviously without the second instance of the OS from a PE or Linux CD/Usb thingy.

Once the OS volume you want to backup/restore is "offline" you can use almost *any* software, as you have "removed" most of the possible issues connected with concurrent access to files, and thus lessened possibilities that something goes wrong (but Murphy's Laws will stiill apply).

On the NT "side" I recommend:

Clonedisk:

http://reboot.pro/8480/

Odin:

http://sourceforge.net/projects/odin-win/

On the Linux side:

http://ping.windowsdream.com/ping.html

If using DOS :w00t:, the mentioned in the other thread Partition Saving

http://www.partition-saving.com/

is possibly the best choice as it twofold threefold fourfold DOS/Linux/Windows/PE ....

so you can practiacally use it from *any* environment.

As said, a product that is interesting and that could be a simpler solution as long as you are going to use as "backup" a disk (actually the standdard approach to backup involves two of them, rotating) that is similar in size (or bigger) is XXCLONE, that can be run from within Windows and that allows (once the "first" clone is made) for very fast "incremental backups":

http://www.xxclone.com/itheory.htm

jaclaz

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...