[SATLUG] Amazon S3

Robert Pearson e2eiod at gmail.com
Thu Jan 31 17:27:52 CST 2008


On Nov 29, 2007 4:57 PM, John Pappas <j at jvpappas.net> wrote:
> I am a little late, but I may have some useful info for those interested...
> On Nov 27, 2007 9:37 PM, David Kowis <dkowis at shlrm.org> wrote:
> > Sean Carolan wrote:
> > > On Nov 27, 2007 8:45 PM,  <aaron_satlug at clamp.net> wrote:
> > >> On Tue, Nov 27, 2007 at 07:57:32PM -0600, Sean Carolan wrote:
> > >>> Is there a way to use rsync to back up your data into S3?
>
> Amazon S3 is a web service only, so there is no native S3 service
> listening for the RSync.  I use InfiniteBits (~5$/mo) for FTP and WWW
> access.  Their "Enterprise" service allows basically anything you
> want, since it runs EC2 instance(s) on your behalf, so a properly
> configured instance can listen for whatever you want.
>
> > >>>  This is my primary interest.
>
> Understandably.  If you find a solution that works, let me know!
> Presumably, you could add an S3 module to rsync to allow the alternate
> target,  I have used jets3t's synchronize (requires Java) to upload
> files via cron.  It is not rsync, but the net is the same.
>
> > S3 is pretty cheap too. I'm considering using it as an offsite backup
> > solution. Probably for no more than $10 a month, including operations
> > and automagicness. Just encrypt all my backups using a gpg key and store
> > them up there. Hard drive failure? no problem ;)
>
> I use it for myself, my business, and 3 of my customers using it.  2
> Customers are windows, and only require that the week's backups be
> uploaded automatically, so I use Super Flexible File Syncronizer
> (www.superflexible.com) running as a service.  I considered using
> JungleDisk and S3Drive, but like the email notification capability and
> flexibility of SFFS.  For myself and my business, I use SFFS
> (Commercial App), JetS3T (Java), Brackup (Perl), S3Fox (Firefox
> Extension), Infinitebits.info (Service), and BucketExplorer, depending
> on the need.
>
> Here is my November Usage (my first "heavy" month):
>  November 1 - November 6, 2007
>                 $0.15 per GB-Month of storage used
>         4.083 GB-Mo     0.61
>                 $0.10 per GB - all data transfer in
>      6.456 GB           0.65
>                 $0.18 per GB - first 10 TB / month data transfer out
> 0.016 GB                0.01
>                 $0.01 per 1,000 PUT or LIST requests
> 129 Requests    0.01
>                 $0.01 per 10,000 GET and all other requests                   9
> Requests        0.01
>
>                                 1.29
>   November 6 - November 30, 2007
>                 $0.15 per GB-Month of storage used                        22.601
> GB-Mo   3.39
>                 $0.10 per GB - all data transfer in
>         84.436 GB       8.44
>                 $0.18 per GB - first 10 TB / month data transfer out    0.084 GB        0.02
>                 $0.01 per 1,000 PUT or LIST requests                      816 Requests  0.01
>                 $0.01 per 10,000 GET and all other requests         15 Requests         0.01
>
>                        11.87
>
>                                     13.16
>
> HTH,
> John


Thanks for the very good information. Very helpful.
An update on Amazon and "Cloud Computing for Everyman".
>From Harry Newton at Technology Investor for Thursday, January 31, 2008
<<http://www.technologyinvestor.com/login/2004/Jan31-08.php>>

Outsource your computing to The Cloud. This is big for computing and
huge for entrepreneurs. A startup walks into my office yesterday. They
have an incredibly sophisticated idea for a new service on the
Internet. I ask "Where will you get all the computing power you need?"
They answer "The Cloud." I look quizzical. They say "Google Amazon Web
Services." The first thing I get is this piece from the present issue
of Forbes magazine:

Why buy computers when you can rent them from Amazon, EMC or Yahoo?
Has Jeff Bezos got a bargain for you.

Like everyone else, the executives at gossipy real estate Web site
Zillow have been anxiously watching housing prices collapse. Hoping to
spice up its offerings to a discouraged consumer, Zillow recently
recalculated the values on 67 million homes over a 12-year period, a
database of figures that took up 4 terabytes of memory. The company
figured it would need six months and millions of dollars to make it
happen. Instead, Zillow ran the job over the Internet, on 500 computer
servers rented from Amazon.com. It took only three weeks and cost less
than $50,000.

"This is a computer-development playground," says Spencer Rascoff,
chief financial officer of 165-employee Zillow.

The next revolution in high tech is taking place inside the "cloud" of
the Internet. Small outfits looking to do lots of computing in a hurry
are not buying hardware anymore; they're renting from established
players that already operate vast networks of cheap computers.
Time-sharing, a concept from the dawn of the computing age, is back
with a vengeance.

Amazon launched its service in March 2006 by renting basic data
storage and has since gone into services like computing and databases.
Storage-hardware giant EMC in October paid $75 million for the
consumer online storage service Mozy, which it plans to expand into an
industrial-strength business, possibly cannibalizing its own hardware
sales. In March Microsoft is expected to offer its own version of
cloud computing, likely aimed at big businesses. Yahoo will move into
the business later in the year. Google has shown no interest in
leasing out its vast infrastructure, but it is challenging Microsoft
with a suite of online applications akin to Office.

The spreading of the cloud darkens the outlook for traditional
hardware makers such as Dell, Hewlett-Packard and Sun Microsystems,
which have already been buffeted by fears of a U.S. recession.

Amazon Web Services has already won over customers such as the New
York Times, Red Hat and SanDisk. Consultancies are also springing up,
selling companies on ways to use Amazon for things like online backup
systems and database clusters to speed a Web site's performance. The
pitch is that if the downturn hits, hiring Amazon is something a
department can do with spare cash and no authorization.

Amazon charges 15 cents for a gigabyte of storage and 10 cents an hour
for a server, services that customers say are up to 90% cheaper than
rental alternatives from computer-hosting companies like Equinix and
Rackspace.

More than saving on hardware purchases, companies like relieving their
tech staffs of maintenance chores. Ooyala is an online video company
that offers thousands of hours of high-definition video through Amazon
Web Services. At Ooyala's usage levels, Chief Technical Officer Sean
Knapp says, it might be cheaper to buy servers, but "this accelerates
the speed of innovation."

It almost certainly accelerates Amazon's historically rocky
profitability. The company divulges almost nothing about its costs or
margins but is said to run its Web Services business on huge networks
of computers costing as little as $300 each. Ten cents an hour adds up
to $876 a year in revenue (assuming nonstop usage). If hardware lasts
two years and if, let's say, electricity and other overhead cost as
much as the hardware, Amazon would have a gross margin of 45%, better
than what it gets on books.

In the last two months of 2007 the number of items stored at Amazon
Web Services grew 40%, to 14 billion units. (Units vary in size from a
couple of bytes to 5 gigabytes, and Amazon keeps the totals secret.)
That's a faster growth rate than in the April-October period. Amazon's
s3 storage service now handles 30,000 requests to its database per
second. "We feel really good about our prospects, both for size and
margins," says Adam Selipsky, vice president at Amazon Web Services.
They expect over the next several years the operation will become a
major business alongside its retail business. "We were always a tech
company: First we applied it to retail, now to this."

Manufacturers like ibm and Sun are struggling to move to the new
model. Sun already offers an online rental system and expects to keep
selling hardware. "We've got to be the infrastructure," says Greg
Papadopoulos, Sun's chief technical officer. If you can't beat them,
arm them.

Go to Amazon's web site and you'll find:

Why Use Amazon Web Services?

Battle-Tested Web Services
Amazon has spent 11 years and over $2 billion building the
infrastructure, technical knowledge, and operational excellence to
operate a world class web-scale computing platform. Amazon Web
Services has now released a variety of web services (programmatic
access to its open APIs) that enable developers to leverage Amazon's
data and robust infrastructure, easily and inexpensively. These
fundamental services allow external developers and businesses to build
their web applications in a reliable, scalable, and cost-effective
manner.

Web-Scale Computing
Amazon Web Services (AWS) provides customers the opportunity to
replace existing infrastructure and scale up or down based on resource
demands. This flexibility allows you to run your business at
"web-scale"--uninhibited by growth in the number of geographic
markets. On average, businesses spend 70% of their time building and
maintaining and worrying about infrastructure, and 30% of their time
focused on the ideas that will propel their business forward.
Web-scale computing is helping to invert the 70/30 ratio, enabling you
to spend your energy creating the difference that will make your
business successful.

Web-scale computing characteristics:

- Elastic capacity both up and down
- Fast response time
- 24/7 availability
- Rock solid reliability

Only Pay For What You Use
With AWS offerings, you only pay for what you use. You dynamically
scale your system up and down, depending on your immediate
requirements and only pay for resources when you need them. This means
there are no upfront costs, so your developers can get started using
web services without huge investment. Developers don't need to give up
equity or incur huge capital expenses, because costs scale along with
usage.

Focus on the Idea
Free up your most valuable resource: time. Don't spend time worrying
about scaling data centers, buying hardware, negotiating lease
contracts, dealing with downed servers, or backing-up customer data.
Amazon Web Services takes care of those issues and does the "heavy
lifting". Shifting the focus from managing infrastructure to driving
your business allows you to concentrate on what matters. This business
approach also levels the playing field so everyone can now compete on
ideas, not resources.

Who's Using AWS?
A growing community of 200,000+ developers, start-ups, and Fortune
1000 companies are building robust applications using our technology.
Browse through our Success Stories to learn how organizations across a
variety of vertical markets are realizing the benefits of Amazon Web
Services.

For more, go to Amazon ---
<<http://www.amazon.com/Why-Use-AWS%3F-home-page/b/ref=sc_fe_l_1?ie=UTF8&node=15763371&no=201590011&me=A36L942TSJ2AJA>>


More information about the SATLUG mailing list