Cold Storage not so cold – Google Nearline v Amazon Glacier

It may have taken a little time but Google has come up with an alternative to Amazon’s Glacier cold storage proposition.

Called Google Storage Nearline it is now available in beta as part of Google’s object storage product, Google Storage.

Google Storage is targeted at businesses and a Google Storage account is required to take advantage of Google Nearline as Nearline is a choice when creating a Google Storage bucket / container.

Google Storage Nearline

Once a bucket is designated as Google Nearline it can be used immediately.

Google positions Nearline as providing “the convenience of online storage at the price of offline storage” and indeed it does with access and retrieval times in the order of around 3 seconds.

Nearline also offers regionality of the  storage bucket (similar to what users can expect from Amazon S3 / Glacier). This allows users to control where data is stored. The regional options include U.S., Europe and Asia. The regional storage buckets are not expected to be fully available until Nearline emerges from beta.

Ultimately Nearline is offering companies a relatively simple, fast-response, low cost, tiered storage solution with not only quick data backup but on-demand retrieval and access.

For users who are already using or aware of Amazon Glacier the major differences are as follows:

Nearline                                     Glacier

1 cent per GB pm                     1 cent per GB pm
($10 per TB pm)                        ($10 per TB pm)

3 second retrieval                      5 hours retrieval
(on demand access)                  (request needed)

Data Redundancy                     Data Redundancy
(multiple locations)                    (multiple locations)

Regional support                      Regional support
3 locations                               7 locations

Google Storage API’s               New Glacier API’s
(use existing)                            (API specific to Glacier)

Egress fees                              Egress fees

Retrieval cost .12 per GB         Retrieval cost .09 per GB

Retrieval 4MB/s per TB            Data delivered in 3 to 5 hours
(after first byte / scales linearly)

Availability 99%                        Availablility 99.99%

Pricing and features are of course subject to to change so always check the links below for latest details:

Amazon Glacier Pricing can be found here.
Google Nearline Storage Pricing can be found here.

Amazon Glacier Whitepaper here.
Google Nearline Whitepaper here.

Ed Snowdon’s email service shuts down – advises not to trust physical data to US companies – what are options ?

It has been a while since we did a post and a lot has happened in that time including the explosion from Edward Snowdon and the PRISM snooping revelations. These have continued to gather momentum culminating  in the email service that Snowdon used, Lavabit, closing. The owner, Ladar Levision, said that he had to walk away to prevent becoming complicit in crimes against the American public. All very cryptic and chilling. He also had this advised that he “would  strongly recommend against anyone trusting their private data to a company with physical ties to the United States.” So what to do if you have data stored on remote servers ?

Well firstly you may not care. The data you are storing may no way be sensitive and that is the key ie. you need a strategy for how you deal with sensitive data and sharing of sensitive data so what can you do ?

1. You could consider encrypting the data that is stored on cloud servers. There are various ways to do this. There are client side tools such as BoxCryptor that do a good job of this, and there are also more enterprise type platform solutions such as CipherCloud and Storage Made Easy that enable private key encryption of data stored remotely . Both can be deployed on-premise behind the corporate firewall.

2. You could consider a different policy entirely for sharing sensitive data. On a personal basis you could use OwnCloud or even setup a RaspBerry Pi as your own personal DropBox or again you could use StorageMadeEasy to create your own business cloud for keeping sensitive data behind the firewall and encrypting remote data stored outside the firewall.

The bottom line is think about your data security, have a policy, think about how you protect sensitive data.

 

Amazon S3 showing elevated error rates

In a recent post CenterNetworks noted that the Amazon S3 service is showing elevated error rates. They noticed that several images were not loading correctly and they heard from multiple CN readers with the same issue on their sites.

They note the issues seem only to be hitting the U.S. Standard centers — other S3 centers including Northern California, Europe and Asia are functioning correctly.

Amazon S3 add RRS – Reduced Redundancy Storage

introduce a new storage option for Amazon S3 called Reduced Redundancy Storage (RRS) that enables customers to reduce their costs by storing non-critical, reproducible data at lower levels of redundancy than the standard storage of Amazon S3. It provides a cost-effective solution for distributing or sharing content that is durably stored elsewhere, or for storing thumbnails, transcoded media, or other processed data that can be easily reproduced. The RRS option stores objects on multiple devices across multiple facilities, providing 400 times the durability of a typical disk drive, but does not replicate objects as many times as standard Amazon S3 storage does, and thus is even more cost effective. Both storage options are designed to be highly available, and both are backed by Amazon S3’s Service Level Agreement.
Once customer data is stored using either Amazon S3’s standard or reduced redundancy storage options, Amazon S3 maintains durability by quickly detecting failed, corrupted, or unresponsive devices and restoring redundancy by re-replicating the data. Amazon S3 standard storage is designed to provide 99.999999999% durability and to sustain the concurrent loss of data in two facilities, while RRS is designed to provide 99.99% durability and to sustain the loss of data in a single facility.
Pricing for Amazon S3 Reduced Redundancy Storage starts at only $0.10 per gigabyte per month and decreases as you store more data. To get started using RRS and Amazon S3, visit http://aws.amazon.com/s3 or learn more by joining our May 26 webinar.
Sincerely,
The Amazon S3 Team

Amazon have introduced a new storage option for Amazon S3 called Reduced Redundancy Storage (RRS) that enables customers to reduce their costs by storing non-critical, reproducible data at lower levels of redundancy than the standard storage of Amazon S3.

It provides a cost-effective solution for distributing or sharing content that is durably stored elsewhere, or for storing thumbnails, transcoded media, or other processed data that can be easily reproduced. The RRS option stores objects on multiple devices across multiple facilities, providing 400 times the durability of a typical disk drive, but does not replicate objects as many times as standard Amazon S3 storage does, and thus is even more cost effective.

Both storage options are designed to be highly available, and both are backed by Amazon S3’s Service Level Agreement.

Once customer data is stored using either Amazon S3’s standard or reduced redundancy storage options, Amazon S3 maintains durability by quickly detecting failed, corrupted, or unresponsive devices and restoring redundancy by re-replicating the data. Amazon S3 standard storage is designed to provide 99.999999999% durability and to sustain the concurrent loss of data in two facilities, while RRS is designed to provide 99.99% durability and to sustain the loss of data in a single facility.

Pricing for Amazon S3 Reduced Redundancy Storage starts at only $0.10 per gigabyte per month and decreases as you store more data.

From a programming viewpoint to enable your storage to take advantage of RRS  you need to set the storage class of an object you upload to RRS. To enable this you set x-amz-storage-class to REDUCED_REDUNDANCY in a PUT request.