Broadgate Big Data Dictionary

Posted on : 28-10-2014 | By : richard.gale | In : Data

Tags: , , , , , , , , , , ,

0

A couple of years back we were getting to grips with big data and thought it would be worthwhile putting a couple of articles together to help explain what the fuss was all about. Big Data is still here and the adoption of it is growing so we thought it would be worthwhile updating and re-publishing. Let us know what you think?

We have been interested in Big Data concepts and technology for a while. There is a great deal of interest and discussion with our clients and associates on the subject of obtaining additional knowledge & value from data.

As with most emerging ideas there are different interpretations and meanings for some of the terms and technologies (including the thinking that ‘big data’ isn’t new at all but just a new name for existing methods and techniques).

With this in mind we thought it would be useful to put together a few terms and definitions that people have asked us about recently to help frame Big Data.

We would really like to get feedback, useful articles & different views on these to help build a more definitive library of Big Data resources.

Analytics 

Big Data Analytics is the processing and searching through large volumes of unstructured and structured data to find hidden patterns and value. The results can be used to further scientific or commercial research, identify customer spending habits or find exceptions in financial, telemetric or risk data to indicate hidden issues or fraudulent activity.

Big Data Analytics is often carried out with software tools designed to sift and analyse large amounts of diverse information being produced at enormous velocity. Statistical tools used for predictive analysis and data mining are utilised to search and build algorithms.

Big Data

The term Big Data describes amounts of data that are too big for conventional data management systems to handle. The volume, velocity and variety of data overwhelm databases and storage. The result is that either data is discarded or unable to be analysed and mined for value.

Gartner has coined the term ‘Extreme Information Processing’ to describe Big Data – we think that’s a pretty good term to describe the limits of capability of existing infrastructure.

There has always been “big data” in the sense that data volumes have always exceeded the ability for systems to process it. The tool sets to store & analyse and make sense of the data generally lag behind the quantity and diversity of information sources.

The actual amounts and types of Big Data this relates to is constantly being redefined as database and hardware manufacturers are constantly moving those limits forward.

Several technologies have emerged to manage the Big Data challenge. Hadoop has become a favourite tool to store and manage the data, traditional database manufacturers have extended their products to deal with the volumes, variety and velocity and new database firms such as ParAccel, Sand & Vectorwise have emerged offering ultra-fast columnar data management systems. Some firms, such as Hadapt, have a hybrid solution utilising tools from both the relational and unstructured world with an intelligent query optimiser and loader which places data in the optimum storage engine.

Business Intelligence

The term Business Intelligence(BI) has been around for a long time and the growth of data and then Big Data has focused more attention in this space. The essence of BI is to obtain value from data to help build business benefits. Big Data itself could be seen as BI – it is a set of applications, techniques and technologies that are applied to an entities data to help produce insight and value from it’s data.

There are a multitude of products that help build Business Intelligence solutions – ranging from the humble Excel to sophisticated (aka expensive) solutions requiring complex and extensive infrastructure to support. In the last few years a number of user friendly tools such as Qlikview and Tableau have emerged allowing tech-savvy business people to exploit and re-cut their data without the need for input from the IT department.

Data Science

This is, perhaps, the most exciting area of Big Data. This is where the Big Value is extracted from the data. One of our data scientist friends described it as follows: ” Big Data is plumbing and that Data Science is the value driver…”

Data Science is a mixture of scientific research techniques, advance programming and statistical skills (or hacking), philosophical thinking (perhaps previously known as ‘thinking outside the box’) and business insight. Basically it’s being able to think about new/different questions to ask, be technically able to intepret them into a machine based format, process the result, interpret them and then ask new questions based from the results of the previous set…

A diagram by blogger Drew Conway  describes some of the skills needed – maybe explains the lack of skills in this space!

 

In addition Pete Warden (creator of the Data Science Toolkit) and others have raised caution on the term Data Science “Anything that needs science in the name is not a real science” but confirms the need to have a definition of what Data Scientists do.

Database

Databases can generally be divided into structured and unstructured.

Structured are the traditional relational database management systems such as Oracle, DB2 and SQL-Server which are fantastic at organising large volumes of transactional and other data with the ability to load and query the data at speed with an integrity in the transactional process to ensure data quality.

Unstructured are technologies that can deal with any form of data that is thrown at them and then distribute out to a highly scalable platform. Hadoop is a good example of this product and a number of firms now produce, package and support the open-source product.

Feedback Loops

Feedback loops are systems where the output from the system are fed back into it to adjust or improve the system processing. Feedback loops exist widely in nature and in engineering systems – think of an oven – heat is applied to warm to a specific temperature and is measured by a thermostat – once the correct temperature is reached the thermostat informs the heating element and it shuts down until feedback from the thermostat says it is getting too cold and it turns on again… and so on.

Feedback loops are an essential part of extracting value from Big Data. Building in feedback and then incorporating Machine Learning methods start to allow systems to become semi-autonomous, this allows the Data Scientists to focus on new and more complex questions whilst testing and tweaking the feedback from their previous systems.

Hadoop

Hadoop is one of the key technologies to support the storage and processing of Big Data. Hadoop emerged from Google and its distributed Google File System and Mapreduce processing tools. It is an open source product under the Apache banner but, like Linux, is distributed by a number of commercial vendors that add support, consultancy and advice on top of the products.

Hadoop is a framework for running applications on large clusters of commodity hardware. The Hadoop framework transparently provides applications both reliability and data motion. Hadoop implements a computational paradigm named map/reduce, where the application is divided into many small fragments of work, each of which may be executed or re-executed on any node in the cluster. In addition, it provides a distributed file system that stores data on the compute nodes, providing very high aggregate bandwidth across the cluster. Both map/reduce and the distributed file system are designed so that node failures are automatically handled by the framework.

So Hadoop could almost be seen as a (big) bucket where you can throw any form and quantity of data into it and it will organise and know where that data resides and can retrieve and process it. It also accepts that there may be holes in the bucket and can patch them up by using additional resources to patch itself up – all in all very clever bucket!!

Hadoop runs on a scheduling basis so when a question is asked it breaks up the query and shoots them out to different parts of the distributed network in parallel and then waits and collates the answers.

Hive

Hive provides a high level, simple, SQL type language to enable processing of and access to data stored in Hadoop files. Hive can provide analytical and business intelligence capability on top of Hadoop. The Hive queries are translated into a set of MapReduce jobs to run against the data. The technology is used by many large technology firms in their products including Facebook and Last.FM. The latency/batch related limitations of MapReduce are present in Hive too but the language allows non-Java programmers to access and manipulate large data sets in Hadoop.

Machine Learning

Machine learning is one of the most exciting concepts in the world of data. The idea is not new at all but the focus on utilising feedback loops of information and algorithms that take actions and change depending on the data without manual intervention could improve numerous business functions. The aim is to find new or previously unknown patterns & linkages between data items to obtain additional value and insight. An example of machine learning in action is Netflix which is constantly trying to improve its movie recommendation system based on a user’s previous viewing, their characteristics and also the features of their other customers with a similar set of attributes.

MapReduce

Mapreduce is a framework for processing large amounts of data across a large number of nodes or machines.

http://code.google.com/edu/parallel/img/mrfigure.png
Map Reduce diagram (courtesy of Google)

Mapreduce works by splitting out (or mapping) requests into multiple separate tasks to be performed on many nodes of the system and then collates and summarises the results back (or reduces) to the outputs.

Mapreduce based on the java language and is the basis of a number of the higher level tools (Hive, Pig) used to access and manipulate large data sets.

Google (amongst others) developed and use this technology to process large amounts of data (such as documents and web pages trawled by its web crawling robots). It allows the complexity of parallel processing, data location and distribution and also system failures to be hidden or abstracted from the requester running the query.

MPP

MPP stands for massively parallel processing and it is the concept which gives the ability to process the volumes (and velocity and variety) of data flowing through systems. Chip processing capabilities are always increasing but to cope with the faster increasing amounts of data processing needs to be split across multiple engines. Technology that can split out requests into equal(ish) chunks of work, manage the processing and then join the results has been difficult to develop.  MPP can be centralised with a cluster of chips or machines in a single or closely coupled cluster or distributed where the power of many distributed machines are used (think ‘idle’ desktop PCs overnight usage as an example). Hadoop utilises many distributed systems for data storage and processing and also has fault tolerance built in which enables processing to continue with the loss of some of those machines.

NoSQL

NoSQL really means ‘not only SQL’, it is the term used for database management systems that do not conform to the traditional RDBMS model (transactional oriented data management systems based on the ACID principle). These systems were developed by technology companies in response to challenges raised by the high volumes of data. Amazon, Google and Yahoo built NoSQL systems to cope with the tidal wave of data generated by their users.

Pig

Apache Pig is a platform for analysing huge data sets. It has a high-level language called Pig Latin which is combined with a data management infrastructure which allows high levels of parallel processing. Again, like Hive, the Pig Latin is compiled into MapReduce requests. Pig is also flexible so additional functions and processing can be added by users for their own specific needs.

Real Time

The challenges in processing the “V”‘s in big data (volume, velocity and variety) have meant that some requirements have been compromised. It the case of Hadoop and Mapreduce this has been the interactive or instant availability of the results. Mapreduce is batch orientated in the sense that requests are sent for processing where they are then scheduled to be run and then the output summarised. This works fine for the original purposes but now the ability to become more real-time or interactive are growing. With a ‘traditional’ database or application users expect the results to be available instantly or pretty close to instant. Google and others are developing more interactive interfaces to Hadoop. Google has Drill and Twitter has release Storm. We see this as one of the most interesting areas of development in the Big Data space at the moment.

 

Over the next few months we have some guest contributors penning their thoughts on the future for big data, analytics and data science.  Also don’t miss Tim Seears’s (TheBigDataPartnership) article on maximising value from your data “Feedback Loops” published here in June 2012.

For the technically minded Damian Spendel also published some worked examples using ‘R’ language on Data Analysis and Value at Risk calculations.

These are our thoughts on the products and technologies – we would welcome any challenges or corrections and will work them into the articles.

 

BROADScale – Cloud Assessment

Posted on : 30-04-2013 | By : jo.rose | In : Cloud

Tags: , , , , , , , , , , , , ,

0

We are well into a step-change in the way that underlying technology services are delivered.  Cloud Computing in its various guises is gaining industry acceptance.  Terms such as Software as a Service (SaaS), Platform as a Service (PaaS), Private Cloud, Hybrid Cloud, Infrastructure as a Service (IaaS) and so on have made their way into the vocabulary of the CIO organisation.

Cloud Computing isn’t new.  Indeed many organisations have been sourcing applications or infrastructure in a utility model for years, although it is only recently that vendors have rebranded these offerings ( “Cloud Washing” ).

With all the hype it is vital that organisations consider carefully their approach to Cloud as part of their overall business strategy and enterprise architecture.

Most importantly, it is not a technology issue and should be considered first and fore mostly from the standpoint of Business, Applications and Operating Model.

Organisations are facing a number of common challenges:

  • Technology budgets are under increasing pressure, with CIO’s looking to extract more value from existing assets with less resource
  • Data Centre investment continues to grow with IT departments constantly battling the issue of power consumption and physical space constraints
  • Time to market and business innovation sit uncomfortably alongside the speed with which IT departments can transform and refresh technology
  • Increases in service level management standards and customer intimacy continue to be at the forefront

Cloud Computing can assist in addressing some of these issues, but only as part of a well thought out strategy as it also brings with it a number of additional complexities and challenges of its own.

Considering the bigger picture, a “Strategic Cloud Framework”

Before entering into a Cloud deployment, organisations should look at all of the dimensions which drive their technology requirements, not the technology itself.  These will shape the Cloud Framework and include:

  • Governance – business alignment, policies and procedures, approval processes and workflow
  • Organisation – changes to operating models, organisation, interdependencies, end-to-end processes, roles and responsibilities
  • Enterprise Architecture – application profiling to determine which applications are suitable, such as irregular / spiky utilisation, loosely coupled, low latency dependency, commodity, development and test
  • Sourcing – internal versus external, Cloud providers positioning, service management, selection approach and leverage
  • Investment Model – business case, impact to technology refresh cycle, cost allocation, recharge model and finance
  • Data Security – user access, data integrity and availability, identity management, confidentiality, IP, reputational risk, legislature, compliance, storage and retrieval processes

The BROADScale service

At Broadgate Consultants we have developed an approach to address the business aspects of the Cloud strategy.  Our consultants have experience in the underpinning technology but also understand that it is led from the Business domain and can help organisations determine the “best execution venue” for their business applications.

Our recommended initial engagement depends on the size, scale and scope of services in terms of the Cloud assessment.

  1. Initial – High Level analysis of capability, maturity and focus areas
  2. Targeted – Specific review around a business function or platform
  3. Deep – Complete analysis and application profiling

At the end of the assessment period we will provide a report and discuss the findings with you.  It will cover the areas outlined in the “Strategic Cloud Framework” and provide you with a roadmap and plan of approach.

During the engagement, our consultant will organise workshops with key stakeholders and align with the IT Strategy and Architecture.

For more details and to schedule an appointment contact us on 0203 326 8000 or email BROADScale@broadgateconsultants.com

Technology Innovation – “Life moves pretty fast…”

Posted on : 25-09-2012 | By : john.vincent | In : Cloud, Data, Innovation

Tags: , , , , , , , , , ,

0

We recently held an event with senior technology leaders where we discussed the current innovation landscape and had some new technology companies present in the areas of Social Media, Data Science and Big Data Analytics. Whilst putting together the schedule and material, I was reminded of a quote from that classic 80’s film, Ferris Buellers Day Off;

“Life moves pretty fast. If you don’t stop and look around once in a while, you could miss it”

When you look at today’s challenges facing leadership involved with technology this does seem very relevant. Organisations are fighting hard just to stand still (or survive)….trying to do more with less, both staff and budget. And whilst dealing with this prevailing climate, around them the world is changing at an ever increasing rate. Where does Technology Innovation fit in then? Well for many, it doesn’t. There’s no time and certainly no budget to look at new way of doing things. However, it does really depend a little on definition.

  • Is switching to more of a consumption based/utility model, be that cloud or whatever makes it more palatable to communicate, classified as innovation?
  • Is using any of the “big data” technologies to consolidate the many pools of unstructured and structured data into a single query-able infrastructure innovation?
  • Is providing a BYOD service for staff, or providing iPad’s for executives or sales staff to do presentations or interface with clients innovation?

No, not really. This is simply evolution of technology. The question is, some technology organisations themselves even keep up with this? We were interested in the results of the 2012 Gartner CIO Agenda Report. The 3 technology areas that CIO’s ranked highest in terms of priority were;

  1. Analytics and Business Intelligence
  2. Mobile Technologies
  3. Cloud Computing (SaaS, IaaS, PaaS)

That in itself isn’t overly surprising. What we found more interesting was looking at how these CIO’s saw the technologies evolving from Emerging, through Developing and to Mainstream. We work a lot with Financial Services companies, so have picked that vertical for the graphic below;

The first area around Big Data/Analytics is largely in line with our view of the market. We see a lot of activity in this space (a some significant hype as well). However, we do concur that by 2015 we expect to see this Mainstream and an increased focus on Data Science as a practice.

Mobile has certainly emerged already and we would expect this to be more in line with the first category. On the device side, technology is moving at a fast pace (in the mobile handset space look at the VIRTUS chipset, which transmits large volumes of data at ultra-high speeds of a reported 2 Gigabits per second. That’s 1,000 times faster than Bluetooth !).

In the area of corporate device support, business application delivery and BYOD, we already see a lot of traction in some organisations. Alongside this new entrants are disrupting the market in terms of mobile payments (such as Monitise).

Lastly, and most surprisingly, whilst financial services see Cloud delivery as a top priority they also see it as Emerging from now through the next 5 years. That can’t be right, can it? (Btw – if you look at the Retail vertical for the same questions, they see all three priorities as Mainstream in the same period).

That brings us back to the question…what do CIO’s consider as Innovation? Reading between the lines of the Gartner survey it clearly differs by vertical. Are financial services organisations less innovative? I’m not sure they are…more conservative, perhaps, but that is to be understood to some degree (see the recently launched Fintech Innovation Lab sponsored by Accenture and many FS firms).

No, what would worry me as a leader within FS is the opening comment from Mr Bueller. Technology and Innovation is certainly moving fast and perhaps the pressure on operational efficiencies, whilst undoubtedly needed, could ultimately detract from bringing new innovation to benefit business and drive competitive value?

There is also a risk that in this climate and with barriers to entry reducing, new entrants could actually gain market share with more agile, functionally rich products and services. We wrote before about the rise of new technology entrepreneurs…there is certainly a danger that this talent pool completely by-passes the financial services technology sector.

Perhaps we do need to “take a moment to stop and look around”. Who in our organisation is responsible for Innovation? Do we have effective Process and Governance? Do we nurture ideas form Concept through to Commercialisation. Some food for thought…

Broadgate Big Data Dictionary Part One

Posted on : 26-07-2012 | By : richard.gale | In : Data

Tags: , , , , , , , , , , , , , , , ,

2

We have been interested in Big Data concepts and technology for a while. There is a great deal of interest and discussion with our clients and associates on the subject of obtaining additional knowledge & value from data.

As with most emerging ideas there are different interpretations and meanings for some of the terms and technologies (including the thinking that ‘big data’ isn’t new at all but just a new name for existing methods and techniques).

With this in mind we thought it would be useful to put together a few terms and definitions that people have asked us about recently to help frame Big Data.

We would really like to get feedback, useful articles & different views on these to help build a more definitive library of Big Data resources.  We’ve started with a few basic terms and next month with cover some of the firms developing solutions – this is just a starting point…

Analytics 

Big Data Analytics is the processing and searching through large volumes of unstructured and structured data to find hidden patterns and value. The results can be used to further scientific or commercial research, identify customer spending habits or find exceptions in financial, telemetric or risk data to indicate hidden issues or fraudulent activity.

Big Data Analytics is often carried out with software tools designed to sift and analyse large amounts of diverse information being produced at enormous velocity. Statistical tools used for predictive analysis and data mining are utilised to search and build algorithms.

Big Data

The term Big Data describes amounts of data that are too big for conventional data management systems to handle. The volume, velocity and variety of data overwhelm databases and storage. The result is that either data is discarded or unable to be analysed and mined for value.

Gartner has coined the term ‘Extreme Information Processing’ to describe Big Data – we think that’s a pretty good term to describe the limits of capability of existing infrastructure.

There has always been Big Data in the sense that data volumes have always exceeded the ability for systems to process it. The tool sets to store & analyse and make sense of the data generally lag behind the quantity and diversity of information sources.

The actual amounts and types of Big Data this relates to is constantly being redefined as database and hardware manufacturers are constantly moving those limits forward.

Several technologies have emerged to manage the Big Data challenge. Hadoop has become a favourite tool to store and manage the data, traditional database manufacturers have extended their products to deal with the volumes, variety and velocity and new database firms such as ParAccel, Sand & Vectorwise have emerged offering ultra-fast columnar data management systems. Some firms, such as Hadapt, have a hybrid solution utilising tools from both the relational and unstructured world with an intelligent query optimiser and loader which places data in the optimum storage engine.

Business Intelligence

The term Business Intelligence(BI) has been around for a long time and the growth of data and then Big Data has focused more attention in this space. The essence of BI is to obtain value from data to help build business benefits. Big Data itself could be seen as BI – it is a set of applications, techniques and technologies that are applied to an entities data to help produce insight and value from it’s data.

There are a multitude of products that help build Business Intelligence solutions – ranging from the humble Excel to sophisticated (aka expensive) solutions requiring complex and extensive infrastructure to support. In the last few years a number of user friendly tools such as Qlikview and Tableau have emerged allowing tech-savvy business people to exploit and re-cut their data without the need for input from the IT department.

Data Science

This is, perhaps, the most exciting area of Big Data. This is where the Big Value is extracted from the data. One Data Scientist partner of ours described as follows: ” Big Data is plumbing and that Data Science is the value driver…”

Data Science is a mixture of scientific research techniques, advance programming and statistical skills (or hacking), philosophical thinking (perhaps previously known as ‘thinking outside the box’) and business insight. Basically it’s being able to think about new/different questions to ask, be technically able to intepret them into a machine based format, process the result, interpret them and then ask new questions based from the results of the previous set…

A diagram by blogger Drew Conway  describes some of the skills needed – maybe explains the lack of skills in this space!

In addition Pete Warden (creator of the Data Science Toolkit) and others have raised caution on the term Data Science “Anything that needs science in the name is not a real science” but confirms the need to have a definition of what Data Scientists do.

Database

Databases can generally be divided into structured and unstructured.

Structured are the traditional relational database management systems such as Oracle, DB2 and SQL-Server which are fantastic at organising large volumes of transactional and other data with the ability to load and query the data at speed with an integrity in the transactional process to ensure data quality.

Unstructured are technologies that can deal with any form of data that is thrown at them and then distribute out to a highly scalable platform. Hadoop is a good example of this product and a number of firms now produce, package and support the open-source product.

Feedback Loops

Feedback loops are systems where the output from the system are fed back into it to adjust or improve the system processing. Feedback loops exist widely in nature and in engineering systems – think of an oven – heat is applied to warm to a specific temperature and is measured by a thermostat – once the correct temperature is reached the thermostat informs the heating element and it shuts down until feedback from the thermostat says it is getting too cold and it turns on again… and so on.

Feedback loops are an essential part of extracting value from Big Data. Building in feedback and then incorporating Machine Learning methods start to allow systems to become semi-autonomous, this allows the Data Scientists to focus on new and more complex questions whilst testing and tweaking the feedback from their previous systems.

Hadoop

Hadoop is one of the key technologies to support the storage and processing of Big Data. Hadoop emerged from Google and its distributed Google File System and Mapreduce processing tools. It is an open source product under the Apache banner but, like Linux, is distributed by a number of commercial vendors that add support, consultancy and advice on top of the products.

Hadoop is a framework for running applications on large clusters of commodity hardware. The Hadoop framework transparently provides applications both reliability and data motion. Hadoop implements a computational paradigm named map/reduce, where the application is divided into many small fragments of work, each of which may be executed or re-executed on any node in the cluster. In addition, it provides a distributed file system that stores data on the compute nodes, providing very high aggregate bandwidth across the cluster. Both map/reduce and the distributed file system are designed so that node failures are automatically handled by the framework.

So Hadoop could almost be seen as a (big) bucket where you can throw any form and quantity of data into it and it will organise and know where that data resides and can retrieve and process it. It also accepts that there may be holes in the bucket and can patch them up by using additional resources to patch itself up – all in all very clever bucket!!

Hadoop runs on a scheduling basis so when a question is asked it breaks up the query and shoots them out to different parts of the distributed network in parallel and then waits and collates the answers.

 

We will continue this theme next month and then start discussing some of the technology organisations involve in more detail, such as covering Hive, Machine Learning, MapReduce, NoSQL and Pig.

 

Broadgate Predicts – Survey Results

Posted on : 26-01-2012 | By : jo.rose | In : Data, General News

Tags: , , , , , , , , , , ,

0

Last month we published 10 Technology Predictions for 2012. We asked for readers to send us their views and also distributed a survey to over 400 clients and associates.

Over 120 people responded, made up of CIO’s, COO’s, Procurement, Technology Change Managers and Subject Matter experts across industries on both the buy and sell side.

 

 

 

 

 

 

Of the responses received, a total of 82% either “Agreed” or “Strongly Agreed” with the predictions. We received a total of 1203 answers to the questions and numerous additional comments.

 

 

 

 

 

 

The responses provided a great insight into the key strategy areas for the coming year. Some common themes were:

  1. Cloud Computing and the continued Commoditisation of IT scored highest in general agreement.
  2. Social Media and Cloud Computing generated the highest number of comments and continue to polarise opinion on the maturity and place, particularly within Financial Services.
  3. Many commented on the current financial constraints within organisations and the impact on the predictions. These were both positive in terms of driving efficiency and negative around funding any change.

If you would like to contribute or obtain a copy of the full report please contact jo.rose@broadgateconsultants.com.

 

 

Broadgate Predicts – 10 themes for 2012

Posted on : 22-12-2011 | By : john.vincent | In : General News

Tags: , , , , , , , , ,

3

As we end what has been an “interesting” year, we look forward to some technology themes for 2012.  These are our views, not analysts or market research firms, and in many cases not revolutionary, but what we have determined during our interactions with clients over the past 12 months and how we see the industry shaping. Let us know what you think.

  1. Cloud Computing gathers pace: as data security blockers reside, particularly in Financial Services, and organisations seek to do more with flat or reduced budgets, companies will look more aggressively to best execution venues for technology services.
  2. Cyber Security issues at the forefront: apart from the high profile incidents in 2011, there has in general been a significant rise in targeted malware attacks across all industries. As we enter 2012 the advanced and persistent nature of attacks will continue, with companies needing to stay vigilant and “one step behind” the cyber criminals.
  3. Commoditisation of IT: we see this a key area in 2012 as developments continue to allow more on demand and utility based compute.  One aspect that will require attention is the organisation as reality hits and companies seek to realign both operating models and the HR impacts of the evolution.
  4. Risk, Regulation and Compliance spending increases: no crystal ball needed for this one.  The impacts of MiFID II, Dodd-Frank, FATCA, Solvency II etc… will continue to drive technology investments higher as a percentage of the overall technology spend portfolio.  However, companies will need to monitor carefully the evolution and practicalities of each to ensure efficient allocation of scarce resources from an already depleted discretionary budget.
  5. Mobility: we will see significant growth both in the mobile payments area, with third party solutions providers increasing market share.  Also, both business and customer end-users will continue to drive the need for always-on data and applications through mobile channels and personal device access.
  6. Business Intelligence: users will require access to management information in a more agile and distributed manner.  A more federated approach to BI data in 2012 will drive improved, enterprise class architectures whilst still empowering users at the organisational “fringes”.
  7. Service Providers leverage PaaS: new offerings from the traditional outsource vendors and service providers will come too market as we see the race to gain market share in the Platform as a Service space. Vertical business solutions will be launched through partnerships with ISV’s and domain experts to leverage the increased demand.
  8. Data Management: we hear a huge amount about the issue of data, be it from a taxonomy, architecture, security, transformation, integration, cleansing, physical or logical perspective etc…  It won’t go away in 2012 and with continued pressure around efficiency it will be a key attention area for 2012.
  9. Portfolio Management: whilst on the agenda for several years, proper portfolio management within organisations has always been a challenge.  Along with the efficiency theme, companies will be seeking more value from technology investments, predictability of outcomes and a swifter remediation ( or cancellation ) of failing initiatives.
  10. Social Media: many organisations dipped their toe in the water in 2011 ( see our prevous blog ).  In 2012 we will see a wider embrace of social media as a channel for improved customer interaction, decision making and a closer tie to financial benefits.

 

 

Is “Cloud Banking” set to explode ?

Posted on : 24-11-2011 | By : john.vincent | In : Cloud, Finance

Tags: , , , , , , ,

2

There are conflicting views on the maturity, positioning and suitability of cloud computing at an enterprise level. Couple that with an increasingly dynamic and evolving marketplace and it is easy to see why it is difficult for organisations to define a roadmap appropriate to their business. What isn’t in doubt is that cloud computing, in whichever form, is changing the landscape of business technology.

However, what is the situation within banking and financial services? I was recently at an event to discuss datacentre innovation with a number of infrastructure managers, architects and consultants in the financial industry. We discussed the fact that demand for IT services outstrips supply and the difficulties that causes internal technology organisations to deal with capacity planning, infrastructure utilisation and optimisation, space and energy requirements. For many, the option of simply building a new datacentre facility to deal with the ebbs and flows of demand is not an option.

We explored techniques and experiences around improving virtualisation and utilisation and also in terms of energy efficiency, with cap-ex and op-ex savings of between 25%-35%. Naturally the conversation moved on to how cloud computing may help in terms of moving power from internally hosted systems to a “best execution” venue.

What was clear was that the starting point for the majority of financial services infrastructure managers was fairly negative in terms of building the higher value cloud models i.e. above Infrastructure as a Service (IaaS) into their technology strategy. Some of the reasons tabled included, “We’re not there yet in terms of maturity”, “Why would we tie ourselves into an external provider?”, “Our data privacy requirements mean that cloud is out for now”, “The regulatory authorities just won’t let us” and so on.

These are, of course, all valid concerns. However, another comment that resonated with me was the anecdote of a business user who, for whatever reason, had decided to turn to Amazon web services for provision of compute power and then simply included it as a line item on their expenses submission.  Not good, but it demonstrates perfectly the tipping point we are at. The question is, how will infrastructure leaders within Financial Services react to a fast-moving market, some of which is driven by user perception and some by the ability to change a service provider?

It is a difficult conundrum. The impact of cloud on the organisation and culture is something we are exploring, but for now let’s look at a few of our predictions for the next two years and why we think that the adoption of cloud with banking and financial services will accelerate.

1) The security issue stops being a blocker:

A key area and one which we believe enterprise security teams will work closely with IT and business users to determine an approach. FS organisations have for years used external providers to manage applications and related data, including Software as a Service, and the same rigour should be applied to allow the appropriate application portfolios to run with external cloud providers (in addition to private).

2) Platform as a Service (PaaS) will see significant growth:

We will see more than just PaaS providers adding multiple distinct environments. In our conversations with technology service providers we believe that many ISV’s will transition their applications to PaaS to provide a more rich set of business services, particularly towards retail banking and corporate systems.

3) The commoditisation of Infrastructure as a Service gathers pace:

The technology discussion will move on from “How do we build and operate the infrastructure?” and start to consider what can be achieved with cloud at a business services level. Some FS technology organisations we speak to are already starting this debate, as the “nuts and bolts” of how to use IaaS are moving into a more commoditised space.

4) Private cloud will continue to expand and provide a “Spring Board” for externalisation:

Having dipped their toe, or perhaps watched others, banks will become more in tune with using private cloud for their IT environments. As budgets continue to be constrained and FS organisations tackle unused or underutilised environments, they will be forced to rethink their IT strategies and shift to adopting scalable cloud infrastructures. In turn, these infrastructures and applications will be considered for transfer to external providers.

5) “Captive” datacentre growth slows and shifts to cloud providers:

This takes us back to the opening discussion on datacentre efficiency. We believe that FS organisations who currently provision their IT environments within ever-expanding datacentres will shift to a “best execution” venue to take advantage of the scalability, on-demand and defined costs of cloud computing. Many companies have already transitioned large portions of their infrastructure to private clouds by introducing virtualised solutions. In parallel with this reduction in internal datacentre footprint, they will need to take advantage of the benefits and economies of scale of public clouds.