Cloudiquity

MySQL is the mainstay of most Cloud Applications (including this WordPress Blog !), however if MySQL has an issue, either through number of connections maxing out, or MySQL being locked and not available it can result in site outages. We’ve seen clients who have ended up with their SQL DB down from a couple of hours to a couple of days before they suddenly realised there was an issue.

To that end we wrote a small script that can be used to automate the restarting of MySQL in such scenarios.

The script is called mysqlrestart.sh and is listed below. You need root access to be able to use it. If you use it and  ever reboot the server you will need to login as root and run nohup ./mysqlrestart.sh  & to restart it.

Set the script to run every 30 seconds using Cron. It will then check for a number of connections and if it cannot get a connection or the number of connections is greater than the number defined (defined as 90 in the example below), it will restart mysql.

#!/bin/bash

SQLCONNECTION_THRESHOLD=90

echo `date`  sqlrestart started >> run.out

while true; do

        sqlconnections=`mysql –skip-column-names -s   -e  ”SHOW STATUS LIKE ‘Threads_connected’” -u root | awk ‘{print $2}’`

        #exclude myself from the number of thread connections             

        sqlconnections=$((sqlconnections – 1))

        echo `date` sqlrestart connections  $sqlconnections  >> run.out

        if [ $sqlconnections -gt $SQLCONNECTION_THRESHOLD ] || [ $sqlconnections -lt 0 ]

        then

                echo `date` restarting mysql server $sqlconnections  >> restart.out

                service mysql restart >> restart.out 2>&1

                echo `date` restart  complete   >> restart.out

        fi

        sleep 30

done

Tags: Cloud, MySQL, restart, Script

We recently started documenting the services we use on a day-to-day basis and it struck us just how much we use SaaS and Cloud Computing services. To that end we thought it would be fun / beneficial to share some of these and how and why we use them:

File Server: SMEStorage

We use SkyDrive (25 GB free) in conjunction with Amazon S3 and an in-house PogoPlug installation to store files. We use SMEStorage Cloud File Server to provide a unified view of all of ur files (we are trialling PogoPlug support with them). This also enables us to assign user / file permissions and governance on the consolidated information stores which is very useful. We can also create collaboration groups across the consolidated information stores for sharing with clients and where the files are stored is abstracted (to the client). We also use the service  to managea nd share files on our iPad (tablet) and Windows Phone 7 clients.

CRM: Zoho CRM

Having used several different CRM systems over time, personally we prefer Zoho. For up to 3 users it is free and adding users and services is reasonably priced. It’s also very easy to change the default templates and to use (using the HTML5 mobile App). SalesForce is synonymous with CRM systems and SaaS, but Zoho is good (cheaper) alternative.

Project Management: Basecamp

Basecamp can be a little expensive but if you manage projects and want to collaborate it is hard  to beat as it’s simplicity and easy to use web interface stand the test of time.

Source Code / Bug Tracker: BitBucket

There are many source code and bug tracking systems, many of them free but we like BitBucket. It’s free for up to 3 users and is a solid source code and bug tracker. For source code editing we use Textastic which enables us to hook into specific source files stored on SkyDrive via WebDav using the SMEStorage CloudDav feature which is enabled when the iPad app is purchased.

Analytics: Google

Google Analytics a essential for tracking website statistics. There are alternatives (such as the fantastic Piwik) but it is hard to beat for ease of use. It is not flawless, the lack of ability to track IP addresses (and therefore do a reverse DNS lookup) is frustrating for example. We use AnalyticsPro on the iPad for mobile access/tracking.

Email: Gmail

Google Apps GMail is a great email system. We’ve used it for years and only have good things to say about it. We backup our Gmail to SkyDrive using the SMEStorage Cloud File Server so that it can be indexed and searchable along with our other files, and of course also for resiliency. For offline access on our iPad’s we use a customised version of Remail that we enhanced for the iPad.

Inbound Lead Tracking: Leadlander

LeadLander enables us to track companies visiting our website, how often they visit, and the new people feature enables us to directly contact leads. It’s a great service and great value at a couple of thousand dollars per year.

Server Monitoring: Server Density & Pingdom & WatchMouse & PagerDuty

Server Density is great for monitoring thresholds on Apache, Server Processes, MySQL connections etc. We plug this into PagerDuty so we can be alerted by phone if major thresholds are breached. Server Density also provide an iOS App for push alerts. Pingdom is used an added check for server outages, and WatchMouse is used to check quality of service for times taken to load pages on a site.

MySQL Admin: PHPMyAdmin

We tend to use the command line but if we need to access MySQL graphically we use PHPMyAdmin.

Call Services: e-Receptionist and Skype and TollFreeforwarding

e-Receptionist is great for virtual teams if you want to route calls to Sales / Support etc and you are not physically in the same location. Skype of course needs no explanation, except that we back up our Skype conversations to SkyDrive using the SMEStorage Cloud File Server so they can be indexed and searchable as we do a lot of communication over Skype. TollFreeForwarding is used to give an international number and then route into the e-receptionist infrastructure.

Online Marketing: Google Adwords and BuySellAds.com and LinkedIn Ads

Google Adwords is synonymous with online marketing and can be a great sales tool if used correctly, and BuySellAds is useful to advertise services to targeted sites using banner ads. LinkedIn Ads is a great way to reach a very targeted audience and in our experience is a great way to compliment any online marketing campaign.

Invoicing Services: BlinkSale

Blinksale is a great, simple SaaS invoicing service that is low cost and very easy to use and administer. There are many others but for simplicity you can’t beat BlinkSale.

Blog: WordPress

There are many others, such as Google Sites, and Tumblr, but there are so many ways to use WordPress and so many plug-in’s that there really is nothing else to compete with it.

Social: Twitter and FriendFeed and Identi.ca and Google Plus and Hootsuite

We use a variety of social sites to push out news. There are of course so many now that it would be impossible to list them all but we covered the major ones. If you use multiple Twitter or social accounts then Hootsuite is a must.

……. and there you have it. The types of SaaS cloud services a distributed virtual company can use to run their business.

Tags: Cloud Computing, Examples, SaaS

The Amazon Cloud has now classed as being FISMA certified. FISMA is an acronym for Federal Information Security Management Act. FISMA sets security requirements for federal IT systems. and is a required certification for US federal government projects.

This is the third set of certifications Amazon has recently announced coming on top of VPC ISO 27001 certification and SAS 70 Type II certification.

The accreditation covers EC2 (Amazon Elastic Compute Cloud), S3 (Simple Storage Service), VPC (Virtual Private Cloud), and includes Amazon’s underlying infrastructure.

AWS’ accreditation covers FISMA’s low and moderate levels. This level of accreditation requires a set of security configurations and controls that includes documenting the management, operational and technical processes used in securing physical and virtual infrastructure, and a requirement for third-party audits.

Other vendors who recently announced FISMA certification recently where Google with Google Apps for Government and Microsoft with the Microsoft’s Business Productivity Online Suite among cloud services (although there was a spat between Microsoft and Google regarding these claims).

Expect to see further certifications as these are a pre-requisite of expansion into lucrative government and private sector contracts as vendors feels more comfortable choosing Cloud resources as commoditisation marches on.

Tags: Amazon, Cloud, FISMA, legislation, Web Services

One of the interesting things that about the last 12-18 months is how the Cloud has put the power into the hands of consumers. What I mean by this is, imagine the following scenario in the world pre-cloud:

“A user buys some software over the Internet or Shrink wrapped. They receive it and install it. It either does not work for them or they cannot figure out how to use it so they basically write off the cash and don’t use the App. End of story.”

Now lets looks at what happens on Cloud:

“The user either buys an Application from an App Store be it desktop or mobile, or a holiday from a holiday store, and then decided either the application is rubbish, does not work for them (or they have not RTFM) or has a bad experience on holiday. The user then use Social networks and/or the review forum on the App Store to comment on the bas experience”

In the latter case this “review” and negative experience puts off other people of trying the App / the holiday / the hotel etc. In some cases it can mean the difference between continually selling product or selling nothing as users look at the last bad review and then move somewhere else to continue their search to buy. One person can have the power to seriously undermine your whole product marketing and application strategy.

Even worse, many of the review forums (Apple’s App Store and Google Marketplace come to mind…) don’t even let you post a counter-review to explain that either the user has got it wrong, or misunderstood, or to genuinely offer to correct a bug. Worse still, some vendors can use a strategy of targeting competitive products to “put people off” purchase. In some cases this has led to the vendors involved seeking out legal action.

So what can you do to protect your product and your reputation ?

Well the first step has already been taken, you are at least thinking about it and conscious of it which is half the battle. What you should do is have your marketing or support team have a strategy that includes:

- Monitoring App Stores and review forums where your product features
- Monitoring social media for keywords about your product or company
- Set up Google Alerts keywords to inform you of keywords about your product and company
- Ensure you check your Twitter messages and also posts on LinkedIn and Facebook pages.  These are easily missed.

When you see responses, always make sure to try and follow up with the user and engage and resolve their issue, even if this means refunding them, Even if a refund seems like the last thing you want to do, offer the refund, it is not worth your reputation. As part of the process try and see if they will change their review, even if it is only to neutral.

The Cloud brings power to the masses in more ways than one and a single user can have a dramatic network effect on your business if you are not careful !

Tags: Cloud, Cloud Computing, Customer Service

Recently we had a customer contact us who had been using Google Checkout to sell an Android application. The customer logged into their Merchant account as they are prone to do each morning to be confronted with “Your merchant Account has been closed” with some templated details about why this could be the case. No specifics, just “we have done this because you may have fallen foul of one of the following reasons”.  You can see the full message here.

This in and of itself is incredible if you think about it, you turn up for work one day, check your business account to find that it’s no longer accessible and no reason has been given, but it gets worse. Within the body of the message the customer is told that any income they have in their account has been refunded, and all access to account details is no longer permitted.

Where we come in is that we were asked if we could help. Well you’d reasonable expect this would be a simple process. Contact Google, figure out what the issues is and resolve it. Right ? Wrong !

Firstly it’s impossible to contact Google by phone to try and resolve this.. It seems that Google has a UK address (this is for a UK conpany) and telephone number but all you get is a recorded message. The address and telephone number is below in full:

Google Payment Limited
Belgrave House
76 Buckingham Palace Road
London SW1W 9TQ
United Kingdom
0207 031 3000

So, we fished about the web to see if we could find a support address for Google Checkout, which we did: checkout-support@google.com. But guess what ? You get an auto-resonder informing you that direct emails are not answered and to use the Merchant help centre, which is basically a set of links, and forums.

We did some more checking on the web and we found the issue is far from a lone issue. Companies invest in using the Google Cloud payment gateway and then at some point in time their accounts is de-activated, any payments for goods they have sent is refunded, and they cannot access any account details, or indeed contact Google direct to see what they have done wrong. This is akin to turning up at your bank to find out you can no longer access your account, your funds have been given back to your customers, despite you having sent them products, and your bank manager will not even entertain talking with you. Seriously we’re not often dumbfounded but in this case we are !

Now we’re not saying that some business may indeed have fallen foul of Google Terms of Service, but this is far from an isolated problem. A quick scan at the Google Merchant forum show many furious customers who have experienced the same issue, and getting in touch with Google at all in regards to Google Checkout issues seems impossible.

So what can you do ? Well it seems all you can do is apply for re-instatement. Further digging found us a link in which you can ask Google via an online contact form as to why the account was suspended. Other than this it seems there is little you can do. Nor is this a new issue. When digging about looking for solutions we found a particular post that is almost 3 years old that read like a litany of the issues our client faces.

Nor is this issue limited to Google Merchant services, PayPal users seem to have experienced very similar issues.

So, the morale here is, choose your Cloud payment gateway carefully. Your business may depend on it ! Stick to reputable merchant service providers and check out their customer service prior to doing any integration work. Make sure you can contact them and fully understand their terms of service.

In a move that has put it into direct competition with competitors such as RackSpace. Amazon has announced that you can now host your website using an Amazon S3 Account. With these new features, Amazon S3 now provides a simple and inexpensive way to host your website in one place at a very cheap price.

To get started, open the Amazon S3 Management Console, and follow these steps:

1) Right-click on your Amazon S3 bucket and open the Properties pane

2) Configure your root and error documents in the Website tab

3) Click Save

Amazon provide more information on hosting a static website on Amazon S3 here.

This is part of a trend that Amazon obviously want to encourage. They recently started an add placement from JumpBox on their free Web Services developers page to offer one click WordPress deployments, amongst other JumpBox offerings.

Tags: Amazon, hosting, Jumpbox, RackSpace, s3

An interesting article in the New York Tines online by Derrick Harris of GigaOM outlined the release of some customers using Clustrix webscale SQL database, namely, Box.net, AOL, PhotoBox, and iOffer.

Clustrix launched in early May with the claim that it’s DB technology, which features MySQL like transactionality and reliability, could scale to billions of entries. All four customers announced by Clustrix in the press release announced that this was the case.

Clustrix provide this as an appliance as an option for online services that do not want to have to invest time and effort in figuring out how to a) shard their data for better performance or b) move to something relatively new such as MongDB, or Cassandra.

Further details (which are worth reading) can be found on the original post on GigaOM.

Tags: Caching, Cloud, Clustrix, GigaOM, NoSQL, Sharding, SQL

As storing data in the cloud becomes increasingly more normal users will increasingly find themselves in the position of needing to access different types of data regularly.  To this end we are starting to see a new breed of applications and services which themselves provide a service that interacts with data stored on the cloud. The challenge is  that services that sell their products or service based on data access are in the position of having to choose which data services to support.

This is further exacerbated in the cloud storage space as their is no ubiquitous API (see our prior post on Amazon S3 becoming a de facto standard interface).

To this end we are starting to see services an applications that themselves are offering interesting aggregations of access to data clouds. We look at a few of these below:

GoodReader, Office2 HD , QuickOffice , Documents to Go, iSMEStorage, iWork:

The iPad,  iPhone, Android have some interesting applications which function on top of existing data clouds. All the aforementioned application work in this way, either letting you view the files (in the case of GoodReader) or letting you view and edit the files (in the case of Office2, QuickOffice, Documents to Go, iWork, and iSMEStorage). The premise is that if you have data stored in an existing cloud then you can load and view or edit it in this tools and store it locally.

Tools such as iWork (which encompasses iPages, iNumbers, and iKeynote) only work with MobileMe or the WebDav standard, although the iSMEStorage App gets around this by enabling you to use iWork as an editor for files accessed through it’s cloud gateway , that can be stored on any number of clouds, using WebDav, even if the underlying cloud does not support WebDav.

In fact some companies are making data access a feature in pricing, for example,  charging extra for increased connectivity.

Gladinet.com and SMEStorage.com :

Both Gladinet and SMEStorage are unique amongst the current Cloud vendors in that they enable aggregated access to multiple file clouds. They essentially enable you to access cloud files from multiple different providers from a single file system.

Gladinet is inherently a windows only solution with many different offerings whereas SMEStorage also has windows software but also has cloud drives for Linux, Mac and also mobile clients for iOS, Android and BlackBerry. Gladinet is  a client side service whereas SMEStorage is a server-based service using it’s Cloud Gateway Appliance ,which is also available as a virtual appliance for VMWAre, XEN etc.

Both offering support a dizzying array of Cloud, such as, Amazon S3, Windows Azure Blob Storage, Google Storage, Google Docs, RackSpace Cloud Files etc, plus many more.

Such solutions don’t just aggregate cloud services but bring the cloud into the desktop and onto the Mobile / Tablet, making the use of cloud data much more transparent.

As data become more outsourced (to the cloud) for all types of different applications and services I expect we will see more such innovative solutions, and applications that give access to aggregated cloud data, and extend the services and tools that are provided by the native data provider.

Tags: aggregation, Azure, Cloud, Cloud Gateway, Docs, Documents to Go, Gladinet, GoodReader, Google, iWork. iSMEStorage, Office2 HD, Quick Office, RackSpace, s3, SMEStorage, Storage

I don’t think anyone would argue that Amazon S3 is the big bear of the Cloud market, both on the virtual cloud infrastructure and the cloud storage side of things. Amazon S3 has more than 102 billion objects stored on it as of March 2010.

As befits a dominant player the interface that Amazon exposes for Amazon S3 is becoming so widely used that it almost becoming a standard with regards to how to connect into Cloud Storage. Many new or existing players in this space already support the interface as an entry point into their Storage infrastructure. For example Google Storage supports the S3 interface, as does the private cloud vendor Eucalyptus with its Walrus offering. Also the on-premise cloud appliance vendor Mezeo recently announced support for accessing their cloud using Amazon S3, as did TierraCloud. There are other Open Source implementations as well such as ParkPlace which is an Amazon S3 clone and bittorrent service that is written in ruby.

Additional to this, the multi-cloud vendor, SMEStorage has implemented an S3 entry point into it’s gateway so that you can use it with normal clouds even where they do not natively support Amazon S3, such as RackSpace, Google Docs, DropBox etc.

So as far as S3 goes it seems you can pretty much access a multitude of  storage back-end’s using this API, which is not surprising as vendors want to make it easy for you to move from S3 to their proposition or they want their proposition to work with existing toolsets and program code. So is it good for cloud in general ? I guess the answer to that is both ‘yes’ and ‘no’.

‘Yes’ from the point of view that standardisation can be a good thing for customers as it gives stability and promotes interoperability. ‘No’ from the point of view that standardisation can easily stifle innovation. I’m happy to say that this is not what is occurring in the cloud storage space as the work around OpenStack and Swift demonstrates.

I think right now, S3 is as close as you will get to a de facto standard for cloud storage API interactions. It probably suits Amazon that this is the case, and it certainly suits consumers / developers. Time will tell how quickly this situations lasts.

Tags: Amazon, API, OpenStack, s3, Swift

If you did not notice the Gawker set of news sites recently has it’s online security compromised. You may not have heard of Gawker but you will probably know of the set of news sites they encompass which includes Gizmodo, Lifehacker, Kotaku, io9 or Jezebel. Over 1.3 million passwords where stolen and uploaded as a 500MB torrent file. Also posted where Gawker’s source code and internal employee conversations. The disclosure of this authentication information led to a viral effect with increased spam attacks, for example, on Twitter being attributed to the breach. Many users use the same web password everywhere so such a breach could leave them exposed on every site where they use the same username and password.

Apparently the passwords where encrypted in the torrent but as Gawker used an outdated encryption scheme they are relatively straightforward to crack. If you have ever registered on any of these sites then and tend to use the same username and password then you should change your username and password anywhere else you have used it on the web. Some sites are already pro-actively forcing you to do this. I receive an email from LinkedIN today that made me go through the lost password security mechanism to reset my account.

So what does this mean for Cloud ? Can one site damage the concept of storing and accessing information on the Cloud ? I think for sure, yes. It will make companies who were reticent about going to Cloud because of security concerns even more reticent, and such a breach has an effect on other sites, and I am sure we have not seen the full fallout of this yet. As for Gawker’s brand, well I think it is hugely damaging, although the web can be a fickle place, it remains to be seen how badly affected the Gawker brand will be. I can imagine potential advertisers do not want to be associated with it.

What can you do to protect yourself ? Well first, for sure change any username/password combos that are the same as the one you registered on this site, and in future consider having a separate username/password combination for each site you register. I create email addresses specifically for a registration for such sites on the web and I file them in KeepPass to be able to remember them. Ulitmately, remember, as a user don’t rely that such sites will protect your data, and as a vendor, revisit your security mechanisms to ensure the next Gawker is not you !

Tags: Cloud, gawker, lifehacker, Security