Meaningful cloud metrics are out there somewhere

There are metrics, and then there are metrics, and with the cloud it sometimes gets to be a bit difficult to decide which metrics are actually important. The following story is, therefore, being used more as a convenient example, rather than as a specific pointer to the efficacy – or otherwise – of any of the participants.

The convenient example concerns FireHost, a hosting services provider, and UK-based Zeta Compliance Technologies. The former has been hosting the latter’s enterprise SaaS applications in its secure cloud infrastructure since the beginning of this year.

In that timeZeta Compliance Technologies, which is part of the Zeta Compliance Group PLC, has seen a 750 percent improvement in application response timewhen compared to its previous cloud hosting provider. Using its own benchmark routine it noted a reduction from aprevious average delay of  4,500 msec. to an average of 600 msec.

This translates to significant time savings for Zeta customers, which include Ministry of Defence and NHS sites, and has a tangible impact on day-to-day business operations. It was also one of the reasons Zeta chose FireHost.

This is excellent as far as it goes, but it begs a serious question as to how relevant a measure is the performance of the hosting company when measured at its end of the communications pipe. That 750 percent reduction is certainly important, but there can be no guarantee that it automatically translates into an overall better response for the user.

For a start, there are usually many hops made as part of the journey from host to user, and back again, and each hop is a potential point of service restriction, and occasional failure. In addition, there can be problems with the users’ own infrastructure that can induce service degradation and, out there in the real world, the continuing problem about how close any user business is to the source of the local loop, as service degradation at that point is still one of the key weaknesses of service performance, particularly in the UK.

While not wishing in any way to impugn Zeta’s own software, there are many instances where, no matter how fast the delivery of the code, the service it is intended to provide for the end user is at the very least poor.

We are still in the early years of developing and building services that meet the real needs of end user businesses, not least because many users are still in the process of thinking through what constitutes the service they require. And we are still in the early years of learning which metrics are relevant to judging whether a service can deliver a meaningful result in an appropriate timescale. This includes the fact that, for most businesses, a `meaningful result’ will normally equate to revenue. The time to cash is becoming an important metric, but the bigger one is the amount.

For example, Zeta Compliance Technologies provides services to organisations with large estates, including educational establishments, high street shops, power stations and healthcare institutions. Zeta’s solutions are used to carry out equipment inspections, risk assessments and other safety checks using barcode scanners linked with its web-based application, ZetaSafe. This collects data online and stores it securely in a database. This simple practice brings a high level of automation to what was once a very time and labour intensive process.

Each day, ZetaSafe processes extremely large amounts of sensitive data for compliance purposes and, as such, needs to be supported by a high performance, secure cloud infrastructure.

This is an application where a number of possible metrics can be seen as potentially important, not least being the reduction in contingency budgets needed to cover missed safety checks or failures of compliance. And when it comes to service delivery this brings the argument of whether reliability is the more important metric than delivery speed.

Selecting the most meaningful metrics is going to be one of the key criteria for assessing cloud services into the future, yet the delivery industry is still some way from identifying them, let alone using them.

It would be interesting to learn readers’ own views on this subject.