Monthly Archives: May 2013

The Future of IT 4: Big Data

big data


Speak to the average IT person about Big Data today and you will get one of three answers:-
–          We don’t have it, that’s just for the large on-line guys!
–          Speak to the BI team, they are responsible for data analytics.
–          We don’t know how to build the business case.

Ask for a definition of “Big Data” and that’s where the fun starts. Some will talk about the classic 3 V’s – velocity, variety and volume, but seldom do you hear anything that gives you a clear idea of neither the value nor the methodologies involved.

I think that this name “Big Data” is a misnomer!  Firstly I’m not convinced that it’s about “Big” I have spoken to people who have gained immense value from understanding their small data. Secondly, I agree with Harper Reed who says it’s not about the Data .. it’s about the Answers!  (Harper was the CTO for the Obama campaign and ran an immense Big Answers project, have a look here for more.)

To me, this thing called Big Data is about understanding what the data has to tell you!

How? Simply by bringing relevant data together to create context, then present that data in a human digestible form.

Bringing data together is where the magic starts, we bring all kinds of data together and the technologies allow us to link, map and/or match this disparate data together. This is very different to the traditional technologies. When people ask why we can’t use the existing technologies I say there are two problems, Relational Databases and Relational Databases.

The first problem with relational technologies is that they are relational! That means you need to analyse the data and structure it before you begin. The result of this is that you have pre-determined the scope and results that you will ever get out of the system. This is not valid in a situation where you don’t know what you will be looking for in six months time.

The second problem with relational technologies is the architecture was built for transaction processing workload. The “Big Data” workload does not fit a TPC-x model at all.  In the end if you attempt to do this work and scale it, it will cost a fortune!

I know that might be too simple for some, but in my simple definition “Big Data” turning data into something that people can use.  To expand on this idea we have to acknowledge that the human mind is better at processing certain types of stimuli, and is limited in the scope of what can be processed.  In 1956 a study was done that showed that the human mind can respond to an average of five, plus or minus 2 stimuli at any one time.  You know that you can stare at a screen full of numbers all day and not be able to detect the trend or an incorrect entry; however you can spot a dead pixel on a HD TV in a moment!  Even relationships have a finite number, known as Dunbar’s number which is 150, above that number and you cannot retain a stable social relationship.

That is why the techniques in ‘Big Data’ allow us to model and visualise the data… to create info-graphics to allow us to grasp the meaning and to build on the knowledge we gain.


We are transitioning to a world which is real-time analytics enabled! (I will talk about this in the future when I bring this altogether.) Unlike today, where everyone has compartmentalised traditional and “Big Data”, much like on-line and batch processing, into two different computing disciplines. The future has to be one where these two worlds work hand in hand or perhaps future architectures incorporate both as a fundamental design.  This is how the large internet companies operate today; imagine if you purchased a book from Amazon and a week later they sent you a recommendation for another book, would you open the e-mail?

So let’s just summarise this down to ‘Big Data’ = Understanding.

Next: Bringing it All together.


The Future of IT 3: Mobility = Connected

Mention “Mobility” to any IT person these days and they go a bit ashen, and the conversation invariably turns to MDM, (mobile device management.)  When you chat to these people you realise that MDM may stand for “Massive Device Mess). There are always areas where the early adopters see the advantage that a new technology provides.  Hospitals have seen how they can un-tether their staff from the office terminal and make them more productive and effective by entering data by the bed side. Restaurants similarly use devices to take orders that are immediately sent to the kitchen. Even airlines have replace pilot’s map cases with tablets, hopefully without the in-air entertainment system installed.

How else have things changed? Let me ask you, do you check your e-mail while in bed? Perhaps you check your e-mail before you check your partner?  You join 71% of the respondents to a recent survey, who said they check e-mail first thing in the morning, while another 17% are on Facebook. Then before we go to sleep 47% of us do a final check of e-mail while 27% make sure we know what our friends are up-to on Facebook.  However in-between… we check our devices while commuting, at work, and even while having a cold beverage after work, we glance at our devices.

The header picture I took on a holiday in Fiji. Being school holidays in Australia the hotel pool was crowded and as I walked I noticed that of the approximately 40 people, only two had paper books, the rest on some device. In-fact there were only two places in the hotel where Wi-Fi was available, a café and the pool!

So the core value of mobility is the fact that we are always connected!

The Future
Let’s expand this idea because it’s not just “us” who are connected, but we are in the process of connecting everything!  At Cisco Live earlier this year the keynote speaker, Carlos Dominguez , demonstrated the Philips Hue, a Wi-Fi light bulb. Taking a picture of the Cisco Live Logo the bulbs changed colour to emulate the sign’s hue!  Now I have to say that when a light bulb is connected we are really getting to a place where everything is connected and the world is rapidly moving to the Internet of Everything.

Just consider the potential of being able to communicate with everyone, while measuring and controlling everything!  As I ‘dad joke’ with my children, “light switches also have an off position, you should try it!” Now consider no light switches, just rooms that light up when you walk towards them, as the GPS in your pocket is communicating your movements to your home control system which is predicting where you are going.

Smart cities are being designed and everyday objects being instrumented, in-order to make city life more efficient and safer. Kevin Bloch, CISCO CTO talks about the ‘electronic’ lamp post. A street lamp post which has multiple devices built into it, from measuring weather conditions to safety cameras and yes a simple light sensors to turn the bulb on and off.

So Mobile = Connectivity

Next: Big Data.

The Future of IT 2: Social = The Crowd!



When Social meets the Enterprise there is one of a few reactions:

A) How can marketing tap into an audience of one billion? or
B) Is it productive to let our people access social media at work?  or
C) Can I use social media to cut my cost of recruiting?

Now these are the obvious uses of the various social sites out there, and they are being leveraged. For example it costs organisations tens of thousands of dollars in commissions when a ‘head hunter’ finds a candidate. However, if an employee refers a candidate it’s more likely to be a successful outcome both in terms of skills and experience as well as cultural fit. Today LinkedIn is becoming one of the leading employment sites on the net.

There is also interesting work being done with Facebook whereby you can assume that people who like similar things are similar. So by analysing what individuals have ‘liked’, may tell us what they would like to buy!

However what is the underlying value of this area? Let me suggest that understanding the social network is not as powerful as leveraging the community it represents.


Let’s explore this idea a little bit. Matt Barrie, BRW entrepreneur of the year 2011, is tapping into the global human capital with his website, (here). The idea is to crowd source expertise on a global basis. By posting a job on the site, interested suppliers bid to do your work. Now you might think that is fine and probably useful for web-site design and coding. However review the categories on the site and you will see that work is being done from Aeronautical Engineering to manufacturing a “Variable Pitch Quadcopter”.

I remember when I was part of a start-up we got a logo designed. It took about 4 months from start to finish and involved an extraordinary amount of time from preparing and briefing an agency to continual refinement until we eventually settled on a final design. Using matt’s site you can put a similar brief up and within hours, people submit their ideas from which you can choose from. Substantially cutting the time and cost and dramatically improving the talent pool that you are drawing from.

We can go further and tap into the knowledge of the crowd! If you haven’t visited the Kahn Academy, (here or downloaded their App), then you are missing a major source of education as their tag line says, “Learn almost anything for free.”

Now it’s not just knowledge and expertise that can be crowd sourced. The now famous case study of the “pebble watch” financing through If you don’t know the story; some guys thought it would be cool to design a watch using liquid-ink technology, (used in e-book readers), to create a smart watch that would show the time and link to your phone. They estimated they needed about $100k to develop it and put the project up on At the end of the period they had $10Million in orders! Today a substantial % of movies, charity drives, theatre productions are being crowd funded.

So where is this going? I would suggest that the power in social is actually the value of the crowd. In the future tapping and leveraging the crowd is going to be essential to the longevity of any organisation.

So does Social = Crowd.

Next Mobile.

Brian Gallagher talks in The Real World about cloud storage

Its all happening at EMC World, and a HD streaming of the keynotes on EMC TV

Andy Sitison

Typed in the field —- Excuse the typos

During Brian Gallagher’s ESD keynote, he took the audience through the changes to the VMAX product line.

Here’s some highlights frm his presentation

– 17 VMAX breakout sessions At EMC World

-VMAX Cloud Edition. During the discussion ran a video on the transformation of Navitare, Accenture spin off for airline industry.

-Vmax cloud , provision faster, 34% lower tco, and provide ITaaS with more consistency than AWS.

-VMAX Clud provides automation, strong use metrics as is sized simply through capacity and or performance needs

– Vmax runs 3x faster other compeition. vmax can optimise data placement in an exclusive way,
across data centers (up to 4).

-2.5 x faster than HP 3par and with replication it doubles its time to respond. Aka “Wicked Bad”

– Brian also talked about Vplex. Which had an interesting data factoid.

-It runs in 52% of the…

View original post 276 more words

The End of the Rein of the Relational Database


Unlike all the others, this time it’s not a spelling mistake I am attempting to be puny, let me explain.

Last week I was facilitating a discussion based on how to build ‘consumer grade’ IT. One of the participants said that it was too expensive to even contemplate building analytics into the day to day operation of their organisation. Well, I challenged, how do all these start up internet organisations build their businesses on real-time analytical processing, no one has ever accused a start up of having deep pockets?

For the past 30+ years the realm of data management and data processing has been cantered on relational databases. The organisation once named ‘Relational Software, Inc.’ reigned supreme as the enterprise database as it perfected the architecture to allow a ‘single streamed’ approach to managing data.
Don’t shout, by single streamed I mean to convey the idea of data which is guaranteed consistent, at all points in time. This is a mandatory requirement when dealing with the traditional structured data of business applications like accounts. The stereotypical bank account… where there must be no possibility of timing or sequence delays which would result in ‘incorrect’ balances.
Now why is this approach limiting the applicability of this technology to the ‘big data’ world? Well firstly the architecture does not scale, both size and performance due to the tight-locking architecture. Hence my pun, attempting to use this model for very large, and non structured datasets is going to rein in your results.

So what’s the alternative? Well it’s the idea of multi-threaded where jobs are split up and executed at the same time, and to make it scalable a concept of loose-locking is used. Here at any single point in time you can’t guarantee consistency, however over time you can. That is a hard concept to get your head around at first. Think about 100 people in a room and you tell one person some news, and ask them to pass it on to other people and ask those to people to pass it on as well. Shortly after you start this process there are a lot of people who haven’t heard the news, but then a little while later everyone has. Same thing here!

So this is one area that Big Data technologies dramatically differ to ‘traditional’. Remember no judgement here it’s a matter of horses for courses! But in Big Data you have to release the reins completely!

The Future of IT 1: Manual to Mental


I believe we are experiencing a fundamental change in the way technology is being used, which will have a profound effect on humanity.  Technology in the past has been concerned with automating and augmenting manual processes, now the focus is on mental processes.

For 60 odd years Information Technology has been applied primarily to taking organisational processes and automating or augmenting them. Computing was applied to helping the accountants keep the books, then to managing resource planning, then to customer relationship management. From purchase orders to rolodex’s, these systems on the whole duplicated what people could do with pen and paper. Job done!

If you have a new way to double the speed at which a purchase order is generated, I don’t think anyone will really care. However if you could help someone make “twice as good” a decision, or help designers design ‘twice as good’ a product, or help find “twice as many” fraud attempts; well now people will be very interested.

The job of automating and augmenting the human mind is now the main game!
(Before we get into the esoteric/academic discussion about automating creativity and emotion.. let just agree not go there and focus on automating mundane tasks and augmenting these higher level functions.)

Begin With The Buzz Words

Lets start at the beginning and what is HOT, right now.. which as we all know is SMAC, (not a miss spelt colloquial term for a drug),  Social, Mobile, Analytics and Cloud.

First let’s get rid of one, Cloud. The reason is that Cloud today is synonymous with Cloud Computing and technical Infrastructure. Although the continual development of the infrastructure into a highly scalable, cost effective and agile utility will underpin everything else here, it does not add to my discussion.  Cloud Computing is the accepted norm, it is the enabler to what I am going to outline.

Now over the next few posts lets think a bit more about Social Mobile and Analytics and where these are heading in terms of the fundamental value and potential uses as these evolve.

Next Social