[SATLUG] NFS Mounts

Richard Maynard richard.maynard at gmail.com
Fri Dec 21 12:28:23 CST 2007


Daniel,


> This led me to wonder what exactly the soft mount option is good for,
> especially considering all of the documentation I found online suggested
> it was only good for data corruption. Additionally, I'm curious as to
>
I would say this depends largely on the nature of the data of the NFS share.
Generally speaking, a hard mount is more reliable and introduces less reason
for concern. However, in some applications there is good reason for a soft
mount, consider the following scenario (which comes from an example of
portal web/app system at EarthLink)

10 - Web Content (HTML, JavaScript, Images) Servers
10 - Web Application (Tomcat) Servers
2 Netapp 960 Filers

NFS is used extensively for storing content and logs so they do not need to
be moved around the network later. There are several file systems, say:

/var/logs/tomcat  (RW, Soft, Netapp 1 - Tomcat Logs, Application Logs)
/var/logs/apache (RW, Soft, Netapp 1 - Apache Logs)
/opt/some/dir/htdocs (RO, Soft, Netapp 2 - Static Web Content)
/opt/someotherdir/appfiles (RO, Hard, Netapp 2 - Java Application)

All transient data is written to local disk. The reason that soft mounts are
used on the first three file systems that are available to their respective
servers is to allow for the greatest service uptime to customers. Looking at
different failure conditions:

Netapp 1 Crashes:  With a soft mount, our application will not stop
responding. Apache and Tomcat will get slower as "disk" writes via the NFS
client take longer, but they will not stop responding. We lose some logging
data, but service is not unavailable to customers.

Netapp 2 Crashes: The content just got wiped out from under apache, and
tomcat lost touch with the application directory, this is going to cause an
outage, and the service will fail, however mount options would not change
that fact.

A switch between the machines and router is flaky: Some logging data to
netapp 1 might be lost, but the service will still respond. Apache will
serve up some 404's when trying to read content if an individual disk access
can not be made, but it will not stop serving the requests that it can.
Tomcat will likely be minimally impacted as it does not need to do a lot of
reads. The threads responsible for whatever reads failing will start to
block, which can cause some 500's to get returned to customers. If all of
the logging mounts had been mounted HARD in this case, apache and tomcat
would start backlogging very soon, java will run out of threads, network
connections will stack up because apache will be waiting on disk access, and
the application will spiral out of control.

There are times where a soft mount is appropriate, it's not always going to
cause corruption and woes for your systems. It just needs to be in the right
place, and for the right reasons.


> why I would be getting timeouts on a gigabit network utilizing jumbo
> frames and mounts read and write size set to 32768. The server side is
> an AMD Sempron 3200+ with 4 SATA drives in a RAID 5 configuration and
> the client side is my quad core desktop. Ideas?
>
What protocol do you use for NFS? Hard mounts in Linux using UDP have caused
our engineering groups some grief because they seem to be prone to blocking.
Using TCP for our mounts in many of these instances has allowed us to work
around whatever the underlying problem was. While, not a fix for the issue,
it can be a workaround until you can find out why UDP traffic is being lost
on your network.


>
> Thanks!
> Daniel
> --
> _______________________________________________
> SATLUG mailing list
> SATLUG at satlug.org
> http://alamo.satlug.org/mailman/listinfo/satlug to unsubscribe
> Powered by Rackspace (www.rackspace.com)
>



-- 
Richard Maynard


More information about the SATLUG mailing list