[SATLUG] Filesystem/RAID advice

Brad Knowles brad at shub-internet.org
Tue Oct 12 23:48:37 CDT 2010

On Oct 12, 2010, at 11:10 PM, Nate wrote:

> Both boards have the ability to run RAID 0, 1, and 10 (The ASUS through one of their PIKE boards).

That's almost certainly not "real" hardware RAID, but more of a software RAID with a little hardware support, and thus is critically dependent on the drivers you've got loaded.  This makes it even less likely that you would be able to successfully dual-boot between Windows and Linux and be able to have them both access that same local filesystem.

If you're spending money to build a decent video production workstation, then spend the money in the place(s) where it will actually do the most good, and make sure to get a real-deal hardware RAID controller.  That will at least remove the question of the cross-OS software RAID support problem.

> The reason that I want to make use of this is because video editing and compositing requires fast hard drives, and I cannot afford to go fully with SSDs.

This is where ZFS really rocks.  With ZFS, you can take a small SSD and dedicate it as the commit log volume, and that's where all of your high transaction rate stuff happens.  All the real data is written to the physical disks, but that doesn't include any of the synchronous meta-data updates, which means your I/O operations per second (IOPS) will skyrocket, and that will also enable maximum system throughput to the physical disks.  ZFS is also cool that it doesn't need a RAID controller of any sort, because it integrates the filesystem storage directly into the disk device storage.

There is a very good reason why Sun designed their Enterprise-grade storage solution on top of OpenSolaris with ZFS, and using SSDs to handle both the read and write caching jobs.  You can do mostly the same, on a much less expensive scale.

Of course, I don't think there is a ZFS implementation for either Linux or Windows.  This is where you could really do yourself a huge favor by building a low-cost dedicated fileserver that is optimized for doing just that one type of job, and then having it serve data via NFS to the Linux side of your dual-boot video production workstation, as well as serving the same data via SMB to the Windows side of your dual-boot video production workstation.

IMO, trying to cram both the fileserver and dual-boot video production workstation tasks into a single machine is a major mistake.

If it wasn't a dual-boot machine, you could get away with cramming everything in.  But these are not OSes that play well together in a full mixed read/write environment, and if you're unwilling to run one or the other under a virtual machine, then your only other real choice is to move the fileserver part of the equation off onto another physical machine.

> I need to use Windows because Photoshop and After Effects do not run under linux, not even with WiNE. Unfortunately, there just aren't any open source alternatives for those apps. So it will be a dual boot box. All religious discussions aside, GIMP is a decent image editor it just cannot replace Photoshop's breadth of useful tools or it's workflow, and there is nothing in the FOSS world that compares to After Effects. Blender does an admirable job as a node-based compositer and the now defunct Jahshaka showed promise, but alas, they cannot replace AE.

Okay, so if the real purpose of the machine is to be a rocking video production workstation, and you are insisting that no Linux software can hold a candle to certain critical pieces of Windows software, then why bother with Linux at all?

Brad Knowles <brad at shub-internet.org>
LinkedIn Profile: <http://tinyurl.com/y8kpxu>

More information about the SATLUG mailing list