jaclaz Posted June 24, 2014 Share Posted June 24, 2014 That (devices been much smarter than what they appear from the outside) is a known (you decide) issue or feature, that has been noticed since the advent of IDE disks and exacerbated by the the "solid state devices", beginning with flash disks and CF cards, but easily extendable to SSD's.The "feature" side is that everything is managed by the hardware+firmware, and as such it is undoubtedly faster and more reliable (as long as the firmware is well programmed/stable). The "issue" side is that since a few years we are the proud owners of "black boxes" which have completely undocumented internal behaviours and that can go "beserk" (by pure chance) in which case your only chances to recover data is through the services of the manufacturer of the "black box" (when and if they provide such services) or through the use of specialized tools that are nothing but (well done and "high quality level") a form of reverse engineering or - in some cases - pure "hacks". A hard disk (or USB bridge, SSD, etc. ) controller is to all effects a rather powerful computer in itself, see this interesting hack:http://spritesmods.com/?art=hddhackthat exposes in an unprecedented way the capabilities of a common (not particularly new) hard disk. Till now we had thus a "dumb" filesystem and a "smart" device. With SSD's we have even "smarter" devices. (at least in theory) If you apply to SSD's a "smart filesystem" there is the risk that the two intelligences will come into a conflict of some kind. From time to time I have nightmares about these families of disks malfunctioning (that hopefully have not been "common" and I believe are not even manufactured anymore):http://www.forensicfocus.com/Forums/viewtopic/p=6560563/or however (hopefully) coinfined to niches like security/intelligence, wtc.http://www.pcworld.com/article/225202/Toshibas_New_Self_Erasing_Hard_Drives_The_Ultimate_in_Data_Security.htmlhttp://storage.toshiba.com/storagesolutions/trends-technology/security-hard-drivesas well as those nightmares about BYOD and remote wiping of devices :http://www.msfn.org/board/topic/163387-byod-ideas-opinions-whatever/http://www.forensicfocus.com/Forums/viewtopic/t=10567/ jaclaz Link to comment Share on other sites More sharing options...
NoelC Posted June 24, 2014 Author Share Posted June 24, 2014 Well, knock on silicon, I've been relying on 4 SSDs in RAID 0 since April 2012 and they haven't even hinted at a single glitch. I am thinking I accidentally coupled the right level of in-box smarts with on-RAID-card smarts with file system smarts. -Noel Link to comment Share on other sites More sharing options...
HarryTri Posted June 24, 2014 Share Posted June 24, 2014 Thanks for the information about storage spaces, they are available in Windows 8 Home (or just Windows 8 in contrast with Windows 8 Pro) but you need some more drives. I suppose that you can use virtual drives for the job too but I don't have much initiative to experiment on it at the moment. Link to comment Share on other sites More sharing options...
NoelC Posted July 9, 2014 Author Share Posted July 9, 2014 Just to follow up, my ReFS formatted drives (one internal, one external) have been working flawlessly, going through terabytes of backup data without a glitch night after night. -Noel Link to comment Share on other sites More sharing options...
jaclaz Posted July 9, 2014 Share Posted July 9, 2014 Good news. jaclaz Link to comment Share on other sites More sharing options...
NoelC Posted July 23, 2014 Author Share Posted July 23, 2014 I've read that the system maintains the ReFS system, including checking for "bit rot" (degradation of data stored on the disk but which is not accessed for a long time). Today I caught the System reading all the files on my backup drive. It appears it was doing a (scheduled?) check for bit rot, just as documented.The activity was not intrusive in any way. Only reason I noticed it was because of some fairly relaxed seeking sounds from my backup drive. My USB connection is capable of carrying about 60 MB/second, based on observations of deliberate copies of large files. Yet this activity was only about 15 to 20 MB/second. It wasn't sending the data anywhere, just reading it. I think it's pretty cool to see the new technology in action. -Noel Link to comment Share on other sites More sharing options...
techswine Posted March 10, 2015 Share Posted March 10, 2015 (edited) This is very informative thank you for the overview, there is surprisingly little good information on REFS on the web.Will the self healing feature of REFS function when it is used on a removable device?Is there any benefit to running REFS on a raid5 array, or would that be entirely redundant?It seems that the storage spaces parity is in some ways superior to RAID in fault tolerance minus the performance. Thanks Edited March 10, 2015 by techswine Link to comment Share on other sites More sharing options...
NoelC Posted March 11, 2015 Author Share Posted March 11, 2015 I should think it'd work fine on an external drive, but honestly I don't know for sure how much if any of the self-healing functionality has been exercised - my ReFS-formatted drives both work as well as the day I started using them, so if there has been any recovery activity, I just haven't noticed. I do glance at my System Event Log from time to time, and I haven't seen anything listed there. That might say there's a lack of management information and control with ReFS - or that I just haven't found it. ReFS management panels might be found in server management packages I don't normally use. Certainly Microsoft didn't / doesn't expect us to run ReFS in Win 8+, even though it runs perfectly as far as I can see. How ReFS would work with RAID 5 et. al... I haven't any anecdotal information, so I can't say. Since the two systems work at different levels it seems as if they should complement each other nicely. Whether there would be gotchas or thrashing situations I can't say. I would certainly want to test it on an array before committing real data to it. One other thing, re: Performance... Both my internal HDD and my external USB drive that are ReFS-formatted have been taking incremental file backup updates for quite a long time now, and they're always nearly full, yet I've not noticed any performance degradation at all. Performance is holding up remarkably well, actually. I'm disappointed to see that there appears to be ZERO advancement of ReFS in Windows 10. -Noel Link to comment Share on other sites More sharing options...
vinifera Posted March 11, 2015 Share Posted March 11, 2015 I'm disappointed to see that there appears to be ZERO advancement of ReFS in Windows 10. AFAIK it is still in testing stagesalso by analysis of other IT people, it seems ReFS is crippled compared to NTFSits only advantage is that self-healing thing that nobody cares about Link to comment Share on other sites More sharing options...
techswine Posted March 12, 2015 Share Posted March 12, 2015 As a digital photographer with terabytes of RAW and JPG2000 photos archived over the years, I began to realize that NTFS isn't as bitperfect as I used to believe.A common phenomenon with JPG2000 and some lossless image compression file formats was distorted stripes in stored images over the years. I wouldn't believe it unless I experienced it myself.It seems like some additional ecc layers are needed for long term storage and from my reading it appears the self healing function of REFS is approaching this.I am on RAID5 after losing a lot of good work but it has it's own inherent vulnerabilities (write hole) which from my reading it appears storage spaces entirely circumvents. Link to comment Share on other sites More sharing options...
TELVM Posted March 12, 2015 Share Posted March 12, 2015 In the Wiki article about ReFS there is this epic line that sounds like quoted from a Marx Brothers' movie: Because ReFS was designed not to fail, if failure does occur, there are no tools provided to repair it. Link to comment Share on other sites More sharing options...
techswine Posted March 12, 2015 Share Posted March 12, 2015 I believe every file system was designed not to fail. Link to comment Share on other sites More sharing options...
NoelC Posted March 12, 2015 Author Share Posted March 12, 2015 My understanding is that the ReFS implementation is supposed to embark on the recovery activities itself, absolving the user from having to mess with it. Whether that's comforting or disturbing... All I can say is on both internal and external drives I haven't encountered any problems yet. I buy decent hardware that doesn't often fail, so mostly my interest was/is the performance. It's not really noticeably better, except as I mentioned it seems not to get as balky when the drive is nearly full (who has a drive that's not normally nearly full?). What I was hoping was that by the time of the next OS (e.g., Win 10) they'd make it possible to format the boot/system volume (C:) with ReFS. At this point I've not fooled with ReFS on SSD hardware yet - I believe I'll do that soon. It will be interesting since when the hardware gets more efficient the differences in the implementation of the file system become a bigger factor in the performance. -Noel Link to comment Share on other sites More sharing options...
jaclaz Posted March 12, 2015 Share Posted March 12, 2015 It will be interesting since when the hardware gets more efficient the differences in the implementation of the file system become a bigger factor in the performance.... or the differences in the actual OS drivers for the filesystems, unrelated , but JFYI :http://www.msfn.org/board/topic/125116-fat16-vs-fat32-vs-ntfs-speed-on-usb-stick/ jaclaz Link to comment Share on other sites More sharing options...
NoelC Posted March 12, 2015 Author Share Posted March 12, 2015 (edited) I do know what you mean. Drivers matter. The RAID controller card I run (Highpoint 2720SGL) was very innovative but got an initial bad rep because of the drivers it shipped with. Basically they had been written in the time of HDDs, not SSDs, so when used with SSDs the performance was essentially dismal. Then they rewrote the drivers and lo and behold the performance skyrocketed. But it was too late. The Internet reviews were out. It was not the thing to buy. All that conspired to make the price initially low and was of course good for those of us who paid attention to the later figures (e.g., here). What I don't know yet is whether ReFS will play well with this controller and its drivers. I'm still short one SSD before implementing my planned expansion, in which I'll have all the drives in RAID 0 and set up an additional partition and data volume over and above the 2 TB volume I have now. I believe I'll set that up, at least for initial testing, to be a ReFS formatted volume and see what happens. -Noel Edited March 12, 2015 by NoelC Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now