Possible to Boot from 4 TB System Volume with BIOS?

[dencorso]

We have really different aproaches to it, but I'd partition something like 1 TB system (backed up weekely, and before any patch Tuesday, by cold forensic imaging) and 3 TB data (backed up daily, incrementally, on a file basis, with XXCOPY). Both back ups held in different external USB 3.0 HDDs (two each: most recent and previous or even and odd). Just my 2¢, of course!

[jaclaz]

We have really different aproaches to it, but I'd partition something like 1 TB system (backed up weekely, and before any patch Tuesday, by cold forensic imaging) and 3 TB data (backed up daily, incrementally, on a file basis, with XXCOPY). Both back ups held in different external USB 3.0 HDDs (two each: most recent and previous or even and odd). Just my 2¢, of course!

WHO are the "we"?

 

I beg to differ.

 

How big is an OS these days?

Pure OS install? 30 Gb?

How much are all the stupid updates all together? 20 Gb?

How much space do all the programs on Earth + their settings BUT NOT data actually need? 100 Gb? 200 Gb?

What else (besides OS+updates+programs+their settings) is actually needed on the "system" (what MS calls "boot") volume? another 100 Gb?

30+20+200+100=350 Gb, add another 150 Gb of "slack space" and you get at the very most 500 Gb.

 

Hint: smaller volumes are faster to dd (if you are going to use dd or similar forensic sound image)

 

And a reminder (just in case), volumes can be mounted to folders inside another (NTFS) volume alright ....

 

jaclaz

[dencorso]

We have really different aproaches to it, but I'd partition something like 1 TB system (backed up weekely, and before any patch Tuesday, by cold forensic imaging) and 3 TB data (backed up daily, incrementally, on a file basis, with XXCOPY). Both back ups held in different external USB 3.0 HDDs (two each: most recent and previous or even and odd). Just my 2¢, of course!

 

WHO are the "we"?

 

When I wrote that the "we" I had in mind was "JoelC and me", because I already did know about his fixation on booting from huge volumes, and I was trying to accomodate that.

 

What I actually do is: using 64 GiB for either 7 x86 ou 7 x64, 40 GiB for XP (usually I have two different such boot partitions), 10 GiB for 98SE, DOS 7.1, grub4DOS, and porteus (either x86 or x64, depending on the machine, from ISO). Each boot partition has its own separate backup set, but I skip backing up those boot partitions not used on a given week, keeping instead the backup from the previous week.  

[jaclaz]

What I actually do is: using 64 GiB for either 7 x86 ou 7 x64, 40 GiB for XP (usually I have two different such boot partitions), 10 GiB for 98SE, DOS 7.1, grub4DOS, and porteus (either x86 or x64, depending on the machine, from ISO). Each boot partition has its own separate backup set, but I skip backing up those boot partitions not used on a given week, keeping instead the backup from the previous week.  

 

Sure , that is a "perfectly normal" setup in a multi-boot environment,  but if you remove the need for the XP and the other Operating Systems, you remain with 64 Gb for Windows 7, let's double it for 8 and double again the result to get the space for all the programs on Earth that you don't have+ huge slack space and you still get around 500 Gb.

I doubt that anyone could find that a "tight" space.

 

I mean, once someone has entered the idea/approach that a single huge system and data volume is not possible (besides not being particularly handy in case of needing a CHKDSK or the like) it would make sense to have ONLY the OS + programs in the smallest  (while  still comfortably) sized volume and have more space for the data on another volume.

Until everything is on the same volume as NoelC pointed out the available space is one "chunk", while when there are two volumes it becomes "two chunks", it is less inconvenient if the "chunks" are 0+3.5 Tb instead of (say) 0.5+3 Tb.

 

I am still missing how exactly NoelC plans to access anything  beyond the LBA corresponding to 2.2 Tb on a MBR scheme, however.

Maybe Windows 8 (and those SSD's and his RAID card(s)) can somehow work in "native" 4K sectors , but if this is the case then there is no need to make two volumes, and I am not at all familiar with a RAID 0 stripe across 8 devices (actually any stripe with more than two devices), so I may well be waaaay off target.

 

jaclaz

[NoelC]

We have really different aproaches to it, but I'd partition something like 1 TB system (backed up weekely, and before any patch Tuesday, by cold forensic imaging) and 3 TB data (backed up daily, incrementally, on a file basis, with XXCOPY). Both back ups held in different external USB 3.0 HDDs (two each: most recent and previous or even and odd). Just my 2¢, of course!

 

Thank you but I back up system images every night which can be restored to bare metal, and also data images every night 2 or 3 different ways.  I don't plan to lower the aggressiveness of my backup strategy.

 

I am still missing how exactly NoelC plans to access anything  beyond the LBA corresponding to 2.2 Tb on a MBR scheme, however.

 

Are you saying that with MBR I can't create two partitions that are each within the 2.2 TB limit on a storage device with a total of twice that size?  If that's the case, I have a misunderstanding and I'm back to wondering how to accomplish this.

 

Of course, this is all still blue sky...  It could turn out that I may not really need more storage nor performance until I get that next workstation, probably within the next 2 years, around the time someone invents one that runs twice as fast as the one I have now.  By then, whatever machine that is will be capable of UEFI and GPT partitioning.

 

-Noel

Edited December 22, 2014 by NoelC

[NoelC]

 

you still get around 500 Gb.

 

I doubt that anyone could find that a "tight" space.

 

You're living in the past a bit...  I'm here, in this thread, because I currently find 800 GB to be approaching the limit of what I consider a "tight" space.

 

Photoshop alone will chew through hundreds of gigabytes of free scratch space just stitching one big panoramic image.

 

Just so it's clear, my current status:

 

 

 

-Noel

Edited December 22, 2014 by NoelC

[dencorso]

Are you saying that with MBR I can't create two partitions that are each within the 2.2 TB limit on a storage device with a total of twice that size?  If that's the case, I have a misunderstanding and I'm back to wondering how to accomplish this.

Absolutely. Both the "LBA of first absolute sector in the partition" and the "Total # of sectors in the partition" are unsigned 32-bit numbers and count sectors (see, for instance, MBR#PTE), so that there are just 2^32 sectors available in total, for an MBR, and that's 2.2 TB or 2 TiB, for 512-bit sectors. Now, if 4KiB sectors are used, then the MBR can count up to 16 TiB. But that usually perplexes current OSes (albeit maybe not 8+ ?). More info from IBM: <link>... and at the Starman's page on MBRs, of course. 

 

Now, while it cannot be done easily with internal HDDs, it can be done with USB (2.0 or 3.0) even on XP, as you can read here.

So, with the proper RAID hardware, maybe you can accomplish it, too... but the sole trick remains using sectors bigger than 0.5 KiB.

[jaclaz]

I thought I was clear enough initially:

The limit in size is still there of course, but it is perfectly possible with a few tricks to boot a GPT disk from BIOS, and what happens later depends on the size of disk sector and on the actual OS.

 

A MBR partition entry still has a field for size that is a double word so anything more than FFFFFFFF or 4294967295 can't be wrtten to it, 

so that anything above 4294967295*512=2199023255040 will still become a suffusion of yellow.

 

So, you need anyway a small "system" (MS terminology reversed, "wrong") or "boot" (logical terminology, "right") volume, and then the OS needs to be able to use GPT disks natively.

 

So, if you still insist in having a monolithic boot and system volume/partition, and you find 800 Gb "tight" (though the "scratch space" as well as the pagefile can be - JFYI - somewhere else), you can make a 1 Gb disk (be it a single device or a striped volume over two devices) MBR partitioned, and assemble the other devices the way you want and make them - say - a huge 3Tb GPT partitioned striped set.

 

jaclaz

[Tripredacus]

You're living in the past a bit...  I'm here, in this thread, because I currently find 800 GB to be approaching the limit of what I consider a "tight" space.

Well gee, no I'm (apparently) the one living in the past, as I have no computer with a boot volume larger than 250GB... and that is only because I had to clone a failing drive and it was the smallest size available. My computer at home only has an 80GB drive for the OS.

[NoelC]

Are you saying that with MBR I can't create two partitions that are each within the 2.2 TB limit on a storage device with a total of twice that size?  If that's the case, I have a misunderstanding and I'm back to wondering how to accomplish this.

Absolutely. Both the "LBA of first absolute sector in the partition" and the "Total # of sectors in the partition" are unsigned 32-bit numbers and count sectors (see, for instance, MBR#PTE), so that there are just 2^32 sectors available in total, for an MBR, and that's 2.2 TB or 2 TiB, for 512-bit sectors. Now, if 4KiB sectors are used, then the MBR can count up to 16 TiB. But that usually perplexes current OSes (albeit maybe not 8+ ?). More info from IBM: <link>... and at the Starman's page on MBRs, of course. 

 

As much as you tried to make that clear, you start with "Absolutely" then proceed to describe that the number of sectors in the partition must be 2^32.  Noting that you did not say "in all partitions".

 

So, please forgive me, but I'm assuming you're saying the following is NOT possible in a practical sense (notwithstanding jaclaz's special backflips) on a machine with BIOS only...

 

• One 4096 GB drive
• Containing two partitions, each 2048 GB
• One of which is a bootable primary partition
• One of which is some other kind of partition comprising the rest of the drive

 

... because the MBR nomenclature specifies LBAs in 32 bits only.

 

Did I get that right?

 

Regarding sizes...  We are only entering the age of huge data and huge performance.

 

-Noel

[NoelC]

I'm sorry for seeming like a dunce on this, but you have to admit the huge amount of information on the subject is daunting, and I've only pushed up to the 2 TiB limit - but not beyond - in all my practical dealings so far.

 

-Noel

[NoelC]

So, if you still insist in having a monolithic boot and system volume/partition, and you find 800 Gb "tight" (though the "scratch space" as well as the pagefile can be - JFYI - somewhere else), you can make a 1 Gb disk (be it a single device or a striped volume over two devices) MBR partitioned, and assemble the other devices the way you want and make them - say - a huge 3Tb GPT partitioned striped set.

 

Unfortunately, either because of varying terminology, or typos, or just my denseness it's NOT clear.

 

Can you put that in practical terms matching my OP, please?

 

In other words, not 3 Tb or TiB or whatever you want to call it, but 4 - i.e. what you'd get by adding up 8 x 512 GB drives in RAID 0.

 

My ideal:  Boot BIOS machine to drive C:, a single 4 TB volume, with all the free space available in one place.

 

Worth a look if "ideal" is not achievable:  Boot BIOS to drive C:, a 2 TB volume, with D:, another 2 TB volume right next door (C: and D: partitions on the same composite RAID 0 "drive").

 

Edit:  I've not thrown out the thought that I could follow your footsteps to boot to a GPT partitioned (composite) drive using your aforementioned trickery if that's the only way to achieve the above.

 

Thanks.

 

-Noel

Edited December 23, 2014 by NoelC

[dencorso]

Are you saying that with MBR I can't create two partitions that are each within the 2.2 TB limit on a storage device with a total of twice that size?  If that's the case, I have a misunderstanding and I'm back to wondering how to accomplish this.

Absolutely. Both the "LBA of first absolute sector in the partition" and the "Total # of sectors in the partition" are unsigned 32-bit numbers and count sectors (see, for instance, MBR#PTE), so that there are just 2^32 sectors available in total, for an MBR, and that's 2.2 TB or 2 TiB, for 512-bit sectors. Now, if 4KiB sectors are used, then the MBR can count up to 16 TiB. But that usually perplexes current OSes (albeit maybe not 8+ ?). More info from IBM: <link>... and at the Starman's page on MBRs, of course. 

 

As much as you tried to make that clear, you start with "Absolutely" then proceed to describe that the number of sectors in the partition must be 2^32.  Noting that you did not say "in all partitions".

 

So, please forgive me, but I'm assuming you're saying the following is NOT possible in a practical sense (notwithstanding jaclaz's special backflips) on a machine with BIOS only...

 

• One 4096 GB drive
• Containing two partitions, each 2048 GB
• One of which is a bootable primary partition
• One of which is some other kind of partition comprising the rest of the drive

 

... because the MBR nomenclature specifies LBAs in 32 bits only.

 

Did I get that right?

 

Regarding sizes...  We are only entering the age of huge data and huge performance.

 

-Noel

 

 

With 512-byte sectors, the maximum size possible for a MBR is 2 TiB because the LBAs count ordinally all the sectors in the whole disk using a single 32-bit number. Hence a 4096 GiB disk is not possible at all with 512-byte sectors. It may still be done with 4 KiB sectors, in some special circumstances, but then it's doubtful whether any current OS, 8+ and 10 included, will be able to boot from it (that is, from a non-512-byte sectored disk MBR). That's all.

[jaclaz]

If you (somehow) put together 8 devices each 512 Gb you get approximately 4 Tb total, so:

 

My ideal: Boot BIOS machine to drive C:, a single 4 TB volume, with all the free space available in one place.

 

The single 4 Tb cannot, as it needs to be GPT (unless you use an additional disk - or other device - to boot or use one of the available methods that you deem too complex or "unsupported")

 

Here is from the mouth of the wolf:

http://support.microsoft.com/kb/2581408

AFAIK exactly the same limits of Windows 7 apply to Windows 8.

 

Worth a look if "ideal" is not achievable:  Boot BIOS to drive C:, a 2 TB volume, with D:, another 2 TB volume right next door.

This is perfectly possible, and BOTH disks (or stripes of SSD's) can be MBR.

The 1+3 earlier proposed would work if the first is MBR and the second GPT.

Also, as hinted before, if you are after having a "single drive letter" you can have "C:\" volume on a 2 Tb disk capable of containing more than 2 Tb , as long as it is OK for you to have it NTFS and the other 2 Tb mounted - say - to a mountpoint *like* C:\Data\.

I hope you have clear this latter possibility of mounting a non-boot volumes to a directory on a NTFS filesytem.

 

jaclaz

Edited December 23, 2014 by jaclaz

[NoelC]

Thanks for the clarifications, though some confusion still remains.  Don't worry about it; I've clearly reached the bleeding edge and will have to derive some of the practical aspects myself. 

 

Jaclaz, I fear using mount points as that would corrupt (or at least complicate) the ability to restore a system image backup.  I don't want to go there for that reason.  And besides, that still would not consolidate the free space into one chunk, which is one of the goals.  It's not about logical organization, it's about maximizing stability and utility.

 

-Noel

Edited December 23, 2014 by NoelC