Cloud Service Delivery – Part 3: Organisational Impact

Posted on : 26-01-2012 | By : john.vincent | In : Cloud

Tags: , , , , , , ,


So far we have looked at Hosted ITSM solutions and End-to-End Service Management relating to cloud based deployment. In our final part we look briefly at the organisational aspects.

Pros and cons of cloud aside, what we do see is an increased awareness and acceptance that for most organisations, a journey towards some form of cloud based computing is either underway or imminent. Whilst technology departments wrestle with this in terms of infrastructure, data, security, financials and the like, one aspect that receives less attention is the impact to the traditional IT organisation. Let’s explore some of this.

Business users are demanding more speed and agility for provision of services in an increasingly competitive market place. We still hear anecdotes from both IT and Business along the line of “Getting a test server takes several weeks to provision…”, “Finally got my login ID and laptop after 10 days…” etc. Familiar ? It’s not a criticism, just an observation ( and they’ll be a lot of colleagues claiming “not here sir” ).

At a recent roundtable we heard of the business user who, fed up with waiting, bought some compute power on his credit card and expensed it.

From the organisation aspect this has a real adverse effect. IT departments have been traditionally configured as internal, captive providers, both in terms of people and assets ( granted, sometimes services are outsourced, but again this is driven from an internal perspective point and often lacks business alignment in structure ). It is therefore very difficult for an internal IT provider to reconfigure itself based on;

  • Operating Model: the capability to shift in terms of flexibility of costs in both baseline and discretionary, offering market comparative / benchmarked technology delivery and economies of scale. Internal organisations are also often still domain aligned, so particularly shared infrastructure services prove challenging from a customer relationship perspective. It often requires external change to optimise these factors and drive efficiencies.
  • Motivation and Desire: job security plays a large part of the challenge. Staff join company technology organisations for a number of reasons, from financial through to technical. Creating and engineering solutions is in the DNA of many teams. Indeed, there are many internal technology departments building huge enterprise class private clouds “on behalf of their business”. Really ?
  • Commoditisation of Technology: the evolution of IT products and services is such that there is a much stronger value proposition for “Buy” vs “Build” and a reduction in configuration and customisation. Changing this ethos and reconfiguring the internal organisation takes some time.
  • Commoditisation of Resource: this is a big issue and often the “elephant in the room”. Business technology innovation and enablement is as important as ever, if not more so. However, the “entry level of expertise” at both a technical and business level, particularly lower down the stack, has reduced. This may cause a few discerning cries, but it is a fact. Is a 30%/40% compensation premium for a desktop engineer in capital markets over someone in retail justified now ?

All of these aspects require careful thinking from an organisational change perspective. Much as with outsourcing capability, there is sometimes the view that simply drawing a line between the retained and transitioned resources is sufficient. It isn’t.

To be successful, the end-to-end operating model should be clearly defined and likewise the roles and responsibilities within that. As more cloud services come on stream, Service Management, Domain / Enterprise Architecture and Commercial & Vendor skills, as opposed to technical and operational, will be more key to maintaining the service integrity and delivering business value. Attention to the training, development and realignment of roles should not be underestimated.

So what about the CIO ? Well we wrote before about how the cloud may elevate the role closer to the business. In the meantime, perhaps we will start to see the emergence of the Chief Service Broker ?


Big Data: Big Society or Big Brother?

Posted on : 26-01-2012 | By : richard.gale | In : Data

Tags: , , , , , , ,


  • In 1948 George Orwell wrote 1984 to highlight the risks to democracy and freedom through the misuse of information and control of society from the state using the ‘Big Brother’ all seeing eye to illustrate this.
  • In the UK David Cameron has stated that he wants to see a ‘Big Society’ where power and responsibility is transferred away from central government and that people are empowered to control and shape their own communities and environment.

Technological advancements often have positive and negative aspects. They can be great enablers to business and further the democratisation of society through wider access to and the ability to use information to increase knowledge. They can also be used to control and analyse information in less benign ways.

‘Big Data’ is a term which encompasses the increasing amounts and speed of data creation and mechanisms for analysing & using that data which we think is already creating new business and social models.

  • What is the story behind Big Data?
  • Why is it important for both technology & business?
  • What will be the impact?

Could it be the end of privacy where information on all locations, activities and interests are recorded and analysed or may it prove to be a breakthrough in advancement for people, society and organisations?

We at Broadgate have been thinking about the business implications of big data for a while now. We think this has great potential to extract, analyst and gain business value from the data and information already within organisations.

Micro loans data explosion
A data explosion visualised – Micro loans made using the Kiva software

What is Big Data?

The term Big Data describes amounts of data that are too big for conventional data management systems to handle. The volume, velocity and variety of data overwhelm databases and storage. The result is that either data is discarded or unable to be analysed and mined for value.

There has always been Big Data in the sense that data volumes have always exceeded the ability for systems to process it. The tool sets to store & analyse and make sense of the data generally lag behind the quantity and diversity of information sources.

Existing relational databases (such as Oracle & SQL-Server) are excellent at processing large terabyte level volumes of structured, transactional data but are not efficient or even capable of storing and making sense of petabyte and beyond with video, image and other loosely formatted information. The database vendor have recognised this and extended their traditional management systems to cater for Big Data, Microsoft’s next version due in March is focussing on dealing with these themes. Other technologies have also emerged to handle large quantities but converting that into useful information is still proving challenging.

At the same time the value of data is becoming more widely recognised. All organisations rely on access and interpretation of data but only recently the ability to trawl through and make sense of huge amounts of data is seen as a key competitive advantage. Amazon do this very well with their suggestions and hints comparing your purchases and searches with many others with similar patterns. Trading organisations are increasingly looking towards the ‘social internet’ as additional tools to gain insight into market sentiment and direction. Google are the masters at guiding searches towards accurate results based on data and previous results.

The amount of data being produced is increasing exponentially. Increasing use of social networks, unstructured and multimedia data are accelerating this growth month by month.

The scope of the internet is expanding too – the ‘internet of everything’ is here with data from mobile, telemetry, social & machine based sources all available to be exploited.

Challenges of Big Data

Why is Big Data a challenge? What are the fundamental aspects which cause so many issues for conventional storage and analytics tools? An example is weather forecasting. It is a complicated set of problems requiring vast amounts of data combined with complex algorithms to calculate future patterns. It is also a dynamic environment where multiple factors change constantly to produce a multitude of different possible outcomes. Weather forecasting has improved dramatically in the last few years, it is very now very good at predicting what is happening now… and in the short term future but becomes far less precise at longer term predictions.

– Volume

Data volumes in organisations are growing 40% year on year and that is data that is currently being stored and managed. Additional data such as CCTV outputs or SCADA sensors (used in telemetry for industrial systems) are generally stored only for a brief period before being overwritten as the cost of storing the volume of data is so high. To extract value the data has to be stored and to be retrievable in order to analyse and summarise the information.

– Velocity

The speed of data inputting and also outputting is increasing. As volumes are rising so too is the velocity. Over the last few years data has accelerated from generally slow and controlled transactions (such as emails, messages, trades etc. ) to high frequency trading transactions, messaging, mobile and video based information flows. These have pushed the limits of IT infrastructure to allow them to pass with minimal latency impacts without even considering the ability to store and utilise the information value in the data.

– Variety

Data has changed from being largely text & number based to include voice, images, music, video. Again the previous generation of relational databases have difficulty managing these data types and file based storage has challenges running analytical examination of them.

– Data Privacy

As more data is captured and analysed there is the potential to go beyond help organisations learn better about their clients’ buying habits and to discover more private information about them.  Mechanisms for controlling and policing this capability need to be built into Big Data plans.

Current Solutions

Big Data issues are have two main challenges. How to capture and store the data and then how to process it and analyse it for its information value.

– Capture

All the traditional Database manufacturers have Big Data offerings. These range from Hardware & Software combinations (such as Teradata and Oracle’s Big Data appliances) to tuned databases such as SAP’s Sybase IQ.

Storage is another key area where a combination of high capacity and high performance. This often includes utilising cheap commodity storage, including cloud based disk alongside SSD type solutions such as Violin memory to allow faster access to the datasets.

Hadoop from Apache has emerged as a contender to capture large volumes of unstructured data. It can deal with huge quantities and can break up the data into manageable sizes and distribute it across a large set of machines to allow efficient processing.

 – Process

Storing the data solves one half of the Big Data problem but mining or extracting the value out of it is the key to success and is building new business concepts and companies.

Search companies such as Google and Yahoo have utilised technology developed to process this data. Their success and revenue depends on the accuracy of their algorithms to find information people want and also to direct them to clients’ products and services.

Hadoop and NoSQL are tools that can process and analyse the volumes of data. They approach from different angles but essentially they remove the rigid structure of relational databases and can utilise distributed processing methods to handle the volumes and complexity of the data sets. From a ‘traditional’ business perspective these tools are in their infancy. As Big Data grows in importance we can foresee opportunities for companies to build more accessible methods and tools to access and process data using these frameworks. The RDBMS manufacturers have Hadoop, NoSQL interfaces and solutions and this will only gather pace as the land-grab for Big Data grows.

There will be increasing usage of sophisticated Business Intelligence tools such as Qliktech’s Qlikview that can manipulate the outputs and provide further analytical value & visualisation of these complex data sets.

What is the Future?

Getting value or business intelligence from these sets of Big Data is key for businesses now. Having the ability to trawl and analyse the data to identify patterns not seen before and build ‘what-if’ models on the data may bring to light new business opportunities or scientific breakthroughs. In the weather forecasting example above, very long term weather forecasting may help us start to comprehend changes in global temperatures and to then understand the likely impact of global warming.


Could Big Data be the end of privacy or the start of a new age of information empowerment… Well probably neither but also a bit of both. It is aiming to provide tools to deal with the increasing amounts of information that are available and there will undoubtedly be new businesses created from the managing and use of this valuable by-product of technology.

Broadgate have created the Broadgate Big Data Strategy to help organisations deal with these challenges. Please contact us if you would like us to help you?

New ways of visualising information are developing to help depict Big Data. This is an exciting topic and will grow in importance over the coming year. We are working with our associates to publish an article on this soon.


Broadgate Predicts – Survey Results

Posted on : 26-01-2012 | By : jo.rose | In : Data, General News

Tags: , , , , , , , , , , ,


Last month we published 10 Technology Predictions for 2012. We asked for readers to send us their views and also distributed a survey to over 400 clients and associates.

Over 120 people responded, made up of CIO’s, COO’s, Procurement, Technology Change Managers and Subject Matter experts across industries on both the buy and sell side.







Of the responses received, a total of 82% either “Agreed” or “Strongly Agreed” with the predictions. We received a total of 1203 answers to the questions and numerous additional comments.







The responses provided a great insight into the key strategy areas for the coming year. Some common themes were:

  1. Cloud Computing and the continued Commoditisation of IT scored highest in general agreement.
  2. Social Media and Cloud Computing generated the highest number of comments and continue to polarise opinion on the maturity and place, particularly within Financial Services.
  3. Many commented on the current financial constraints within organisations and the impact on the predictions. These were both positive in terms of driving efficiency and negative around funding any change.

If you would like to contribute or obtain a copy of the full report please contact