Storage Vendors go for broke with OpenStack Swift Storage

openstack logoOpenStack, the open-source on-premise alternative to Amazon S3 is heading into 2015 with a vast mount of momentum. VC’s are falling over themselves to invest in OpenStack related companies and there seems to be genuine enterprise momentum.

The OpenStack story kicked off in 2010 and was initially a combined project between Rackspace and NASA. Fast forward to 2015 and it is managed by the OpenStack Foundation which is a non-profit corporate entity that was established in September 2012 as a means to promote OpenStack software.

Most people may know OpenStack primarily due to it’s infrastructure as a service (IaaS) solution, but it also has an object Storage solution, called ‘Swift’ (not to be confused with Apple’s new programming language, also confusingly called ‘Swift’) which also has garnered a momentum of its own.

Object Storage is a type storage architecture that manages data as objects unlike other storage systems which either manage data as a file hierarchy or as blocks within sectors and tracks (block storage).

The advantages of object storage architectures is that they offer unlimited scalability with a lower emphasis on processing and they offer access using Internet protocols (REST) rather than storage commands.

There is a momentum around Object Storage companies that include such commercial vendors as CleverSafe, Cloudian, Amplistore and Scality.

Vendors who are offering an OpenStack Swift distro as part of their offering include:

HP (Helion Content Depot)
IBM (Cloud Manager with OpenStack)
SoftLayer (Now owned by IBM)
SUSE Cloud
Ubuntu OpenStack
RedHat OpenStack
VMWare OpenStack

As an example of the sums of money involved, Mirantis recently closed a round for $100 million and SwiftStack a round for $16 million, taking both company to total investments of $120 million and $23.6 million respectively. IBM also purchased SoftLayer for a reputed $2 billion. It’s clear that VC’s and Software vendors see something special in OpenStack.

Amazon Web Services may rule when it comes to public cloud but a recent survey sponsored by GigaOM gave results indicating that half of private clouds deployed where OpenStack based.

OpenStack, like Amazon Web Services, is primarily supplied with REST API’s and toolkits  that developers can use to interact with the OpenStack infrastructure. As with AWS this creates opportunities for vendors at the Application level to provide Apps and tools.

Storage Made Easy is a company that has already make an impact on the OpenStack community with its Enterprise File Share and Sync product offering, which has been optimized for OpenStack Swift. The company, itself a startup, already has a growing number of service providers and customer using its enterprise application in conjunction with OpenStack Swift, and has partnered with a number of the key players listed above in a strategy focused around taking advantage of OpenStack’s growth.

Other companies are treading the same path and this itself creates an eco-system of enterprise ready Applications ready to take advantage of OpenStack’s foothold in the Enterprise to grow or to be acquired.

Of course, with OpenStack being an open-source initiative it is not just commercial Apps that have sprung up around OpenStack. There are  Open Source Applications such as Swift Explorer and CyberDuck, but strangely, given the Open Source root of OpenStack there seems to be more commercial offerings rather than open source offerings.

All in all OpenStack is an initiative that is in its ascendancy. It used to be said that OpenStack was more hype than reality but as we head into 2015 the money men have placed their bets and they tend to bet on reality rather than hype.



Ed Snowdon’s email service shuts down – advises not to trust physical data to US companies – what are options ?

It has been a while since we did a post and a lot has happened in that time including the explosion from Edward Snowdon and the PRISM snooping revelations. These have continued to gather momentum culminating  in the email service that Snowdon used, Lavabit, closing. The owner, Ladar Levision, said that he had to walk away to prevent becoming complicit in crimes against the American public. All very cryptic and chilling. He also had this advised that he “would  strongly recommend against anyone trusting their private data to a company with physical ties to the United States.” So what to do if you have data stored on remote servers ?

Well firstly you may not care. The data you are storing may no way be sensitive and that is the key ie. you need a strategy for how you deal with sensitive data and sharing of sensitive data so what can you do ?

1. You could consider encrypting the data that is stored on cloud servers. There are various ways to do this. There are client side tools such as BoxCryptor that do a good job of this, and there are also more enterprise type platform solutions such as CipherCloud and Storage Made Easy that enable private key encryption of data stored remotely . Both can be deployed on-premise behind the corporate firewall.

2. You could consider a different policy entirely for sharing sensitive data. On a personal basis you could use OwnCloud or even setup a RaspBerry Pi as your own personal DropBox or again you could use StorageMadeEasy to create your own business cloud for keeping sensitive data behind the firewall and encrypting remote data stored outside the firewall.

The bottom line is think about your data security, have a policy, think about how you protect sensitive data.


The rise of the Cloud Data Aggregators

As storing data in the cloud becomes increasingly more normal users will increasingly find themselves in the position of needing to access different types of data regularly.  To this end we are starting to see a new breed of applications and services which themselves provide a service that interacts with data stored on the cloud. The challenge is  that services that sell their products or service based on data access are in the position of having to choose which data services to support.

This is further exacerbated in the cloud storage space as their is no ubiquitous API (see our prior post on Amazon S3 becoming a de facto standard interface).

To this end we are starting to see services an applications that themselves are offering interesting aggregations of access to data clouds. We look at a few of these below:

GoodReader, Office2 HDQuickOfficeDocuments to Go, iSMEStorage, iWork:

The iPad,  iPhone, Android have some interesting applications which function on top of existing data clouds. All the aforementioned application work in this way, either letting you view the files (in the case of GoodReader) or letting you view and edit the files (in the case of Office2, QuickOffice, Documents to Go, iWork, and iSMEStorage). The premise is that if you have data stored in an existing cloud then you can load and view or edit it in this tools and store it locally.

Tools such as iWork (which encompasses iPages, iNumbers, and iKeynote) only work with MobileMe or the WebDav standard, although the iSMEStorage App gets around this by enabling you to use iWork as an editor for files accessed through it’s cloud gateway , that can be stored on any number of clouds, using WebDav, even if the underlying cloud does not support WebDav.

In fact some companies are making data access a feature in pricing, for example,  charging extra for increased connectivity. and :

Both Gladinet and SMES are unique amongst the current Cloud vendors in that they enable aggregated access to multiple file clouds. They essentially enable you to access cloud files from multiple different providers from a single file system.

Gladinet is inherently a windows only solution with many different offerings whereas Storage Made Easy also has windows software but also has cloud drives for Linux, Mac and also mobile clients for iOS, Android and BlackBerry. Gladinet is  a client side service whereas SME is a server-based service using it’s Cloud Gateway Appliance ,which is also available as a virtual appliance for VMWAre, XEN etc.

Both offering support a dizzying array of Cloud, such as, Amazon S3, Windows Azure Blob Storage, Google Storage, Google Docs, RackSpace Cloud Files etc, plus many more.

Such solutions don’t just aggregate cloud services but bring the cloud into the desktop and onto the Mobile / Tablet, making the use of cloud data much more transparent.

As data become more outsourced (to the cloud) for all types of different applications and services I expect we will see more such innovative solutions, and applications that give access to aggregated cloud data, and extend the services and tools that are provided by the native data provider.

Is Amazon S3 becoming a de facto standard interface ?

I don’t think anyone would argue that Amazon S3 is the big bear of the Cloud market, both on the virtual cloud infrastructure and the cloud storage side of things. Amazon S3 has more than 102 billion objects stored on it as of March 2010.

As befits a dominant player the interface that Amazon exposes for Amazon S3 is becoming so widely used that it almost becoming a standard with regards to how to connect into Cloud Storage. Many new or existing players in this space already support the interface as an entry point into their Storage infrastructure. For example Google Storage supports the S3 interface, as does the private cloud vendor Eucalyptus with its Walrus offering. Also the on-premise cloud appliance vendor Mezeo recently announced support for accessing their cloud using Amazon S3, as did TierraCloud. There are other Open Source implementations as well such as ParkPlace which is an Amazon S3 clone and bittorrent service that is written in ruby.

Additional to this, the multi-cloud vendor, Storage Made Easy has implemented an S3 entry point into it’s gateway so that you can use it with normal clouds even where they do not natively support Amazon S3, such as RackSpace, Google Docs, DropBox etc.

So as far as S3 goes it seems you can pretty much access a multitude of  storage back-end’s using this API, which is not surprising as vendors want to make it easy for you to move from S3 to their proposition or they want their proposition to work with existing toolsets and program code. So is it good for cloud in general ? I guess the answer to that is both ‘yes’ and ‘no’.

‘Yes’ from the point of view that standardisation can be a good thing for customers as it gives stability and promotes interoperability. ‘No’ from the point of view that standardisation can easily stifle innovation. I’m happy to say that this is not what is occurring in the cloud storage space as the work around OpenStack and Swift demonstrates.

I think right now, S3 is as close as you will get to a de facto standard for cloud storage API interactions. It probably suits Amazon that this is the case, and it certainly suits consumers / developers. Time will tell how quickly this situations lasts.

Research shows that most small businesses use over 3 information Clouds

London, United Kingdom – SMEStorage, the multi-cloud data access provider, polled 1000 of its users to understand how many information clouds they use, and also to understand the challenges that they face.

Of those who responded it was found that the average number of file based storage clouds that were used were three. The breakdown of these were typically two for personal use and one for business use. Of the two for personal use, Google Docs and Amazon S3 were the most popular with Microsoft SkyDrive and Amazon S3 the second most popular combination. Amazon S3 was the most popular storage cloud used for business. Google Docs was the most popular used for documents closely followed by Microsoft SkyDrive.

Of those polled 91% used some form of file server, file store, or stored data only on their PC or laptop. The biggest challenges identified for moving this data to the clouds were availability, security, and speed of access.

Also the majority of users polled had two or more email clients. One or more for personal use and one for work. Asked how much file based information they believed was stored in their email clients over 70% of respondents answered that the amount of file information stored in email was ‘substantial’ and ‘important’.

The majority of users used at least one social network cloud with 45% using two or more and 37% using at least two. The most popular social network with respondents was LinkedIn, followed by Facebook. The most popular combination was LinkedIn and FaceBook with the second most popular combination being LinkedIn and Twitter. Evernote was also used by 27% of respondents.

The most important information stored in Social network clouds that respondents identified was contact information, with the second most important information being identified as photos.

Only 2% of respondents had attempted any form of backup of social network information with 67% believing it was important to do. Contact information was outlined as the most important thing to back up.

The top challenges that respondents identified for the use of Cloud for the majority of their data were: Desktop tooling / ease of use; ‘On the move’ access from different devices; Security and Safety of data; Availability and speed.

SMEStorage has over 100,000 users using it’s multi-cloud Data Access platform and conducted the research to understand how it can best serve it’s users for forthcoming feature enhancements.

SMEStorage already provides the ability to view file clouds, including email clouds, in an aggregated view from a ‘Virtual Cloud File System’ either directly from a web browser, or from virtual drives on either Windows, Mac or Linux Desktop’s.

SMEStorage also provides clients for iPad, iPhone, Android, and BlackBerry. The SMEStorage CloudDav service which enables WebDav over clouds that do not natively support WebDav also enables integration with other smart devices that have native WebDav support, such as the Nokia N8 smartphone.

Additionally SMEStorage also currently enables the backing up of Social network contacts as well public/private file encryption over any storage clouds to enhance the security of data.

Future feature enhancements that SMEStorage intends to roll out include the ability to use the Amazon S3 APi as an entry point for all supported storage clouds; Support for backing up Evernote, Twitter and Instapaper clouds; Support for new storage clouds that will include Google Storage and PogoPlug; Richer desktop and sync clients for all desktop operating systems.