[SATLUG] 67 TB for less than $8K

Brad Knowles brad at shub-internet.org
Tue Oct 9 22:48:16 CDT 2012

On Oct 9, 2012, at 8:03 PM, Bruce Dubbs <bruce.dubbs at gmail.com> wrote:

> You forgot some parts:
> This section is meant to be tongue-in-cheek since it should be clear by now that we have actually gone out and built one of these things (and had a blast doing it, pictures to follow …).

No, I didn't forget that part.  I read it and understood it and I comprehend why the company that employs Chris chose to build Backblaze pods, and how they got a customer to pay for it all.

I really did read all of the articles in that series before I quoted from any of them.

>>> • The system uses a single disk for hosting the operating system
> Setting up a RAID wouldn't be hard.

True, but that's not in the original design.

> • The system has no hardware RAID capability
> What does HW RAID offer over SW RAID?  It may offer better HW, but there isn't a difference between a program loaded out of firmware or off a disk.  I slightly better MB with more PCIe slots or evne a good on-board RAID capability would fix that.

SW RAID works best so long as nothing goes wrong.  But when the effluent hits the fan and you're suffering a two-disk failure on a RAID-6 array, proper HW RAID will slow down somewhat but otherwise the device should keep running, whereas SW RAID will simply go nutzy-Fagin on you and the performance will drop through the toilet.

BTDT.  Hardware RAID is more expensive, but it doesn't have to be that much more expensive.

But as you said, it all depends on what you want/need.

> • The system only has 2 GigE network interfaces
> That can be fixed with not too expensive HW.

Don't need to.  Not in the Backblaze SATA-to-SATA design.  They can't get much more than a single full Gig-E bandwidth even when they are doing pure internal disk-to-disk writes.

If single Gig-E is all you want/need, then you're already fine.

> • If you build this yourself totally DIY you
>  will be required to create custom wiring harnesses
> It is a DIY project.

Re-read Chris' comments about the horrors of building your own custom wiring harnesses, as well as some of the other comments I referenced on that same subject.

Also re-read Chris' comments about how hard it is to get some of the parts at all, if you're doing a DIY build.  And for the parts you can get for a DIY build, pay attention to his comments regarding how much all that will actually cost you, versus how much Backblaze advertises.

> • Any monitoring or health status reporting tools will have
> to be built, installed and configured by hand
> When we designed and built a jet engine testing system for the Air Force (TF-39, F-100, Gas Turbines), we did that and a lot more.  Using government engineers saved is 10's of millions of (1985) $ and it was better than anything the commercial marketplace had to offer.

So, you want to build your own replacement for Nagios, Sensu, AssimMon, or one of the other various monitoring tools?

Most people I know are horrified by the amount of work that goes into just taking one of these tools and installing and configuring it, much less trying to write your own from scratch.

But if you want to do that, please feel free.

> At a far lower price.  If you want everything done for you, you have to pay big bucks.

And now we get back to the issue of all those hidden costs.

If your time costs nothing, and you lose nothing of value if you are down for a significant period of time, and there is nothing more valuable that you could be doing with your time and effort, then by all means feel free to create a multi-terabyte array of 1.44MB floppy drives.

It's the hidden costs that most people ignore when they look at a project like this. That's all I'm trying to highlight.

If you're fully aware of all the hidden costs, and you're okay with all the hidden costs, then I think we're both done here.

Brad Knowles <brad at shub-internet.org>
LinkedIn Profile: <http://tinyurl.com/y8kpxu>

More information about the SATLUG mailing list