Monthly Archives: January 2014

Don’t Shout I’m Back – My Big Data, Social and Mobile Holiday

After committing to this blog and posting just two posts, I went silent. We’ll I’ve been on holiday, two weeks of beach, reading and the Australian Open… all ready for the year to come!
Do you ever stop and think about how different life is now compared to the ‘way it used to be’. Let’s just compare my two weeks of to just 15 years ago. How the nature of data, information and access is changing the way we live!
To start we decided rather late in the piece to go away, being peak season we were expecting availability to be scarce and prices to be extortionist. 15 years ago, (then), my wife would have taken to the phones and after several hours we would have been in a position to make a decision, in this case within 15 minutes we reviewed the options and had a booking with the deposit paid within 20! So welcome to the age of the Internet… but that is not that interesting. What was interesting, (and unfortunate for the vendor), is that we could see this particular set of units did not have a high occupancy and negotiated a much ‘fairer’ price. Sometimes we ignore the price transparency that we enjoy today!
I read a great deal but it’s all ‘work’ related, so I’m lost when it comes to what’s worth reading for entertainment. For me sitting on a beach is about being fully covered and protected while reading. (For people who don’t know me, I’m the kind of person who burns, I don’t tan.) In the past this would mean getting recommendations from friends and an extended trip to the book store/ newsagent to find stuff to read… now it was a quick ‘trip’ to Amazon books. After selecting one book… the recommendation engine suggested a few others, which also got loaded onto the Kindle. Once again nothing new here, but sitting on the beach I was expecting to be the only nerd, however as I scanned the beach at least 80% of the people reading were using a device, rather than filling pages with sand!
In the past I’ve managed to watch the major games of the Australian Open, but this year I got to watch the championship build through the two weeks, a totally different experience. It seems to me like the first week are the warm-up matches for the top players. However, as they move into the second week there is a perceivable shift in play. Now, for better or worse it seems like tennis is moving to look more like baseball, heavy heavy statistics! You will notice on TV every few games the commentators are throwing some fact and stat at you. However if you went multi-screen and also participated in the online experience you could get overwhelmed with the information available.
I’ve discussed the ‘slamtracker’ that Tennis Australia has on their website that tracks the way the opponents are playing and how they are achieving their ‘key play objectives’. The idea is that each player has a certain style of play and as such to beat them you have to adapt your style to overcome their strengths and exploit their weaker side. (For example Nadal served almost exclusively to Federer’s backhand!) Now they have added a social monitoring as well, to me more for interest than predicting the outcome of the game. So while the game is being played you can not only see all the statistics in-front of you, you can see how well they are sticking to a winning game strategy and see what people are thinking. Like the massive drop in attitude towards Nadal as he left the court during the final, and expressed with the boos from the crowd when he got back!
All in all…Tennis Australia today is where Social, Mobile and Big Data come together in one place!
So sitting on the beach, reading my kindle with an eye on the tennis… did mean a beach holiday was very different to what I recall just 15 years ago!


The Storage World Accelerates – Hardware or Software


I just wanted to say I told you so! (Thanks for that, I do feel better now). But don’t go away, I have a way for you to keep one of your NY resolutions!

Last year I said that “Storage was not Snorage” here and was I right?

Well at the end of the year I found this article, “The 10 Coolest Storage Startups of 2013” (here), isn’t that great? Firstly who would have dreamt of a headline about storage a couple of years back, and secondly they had to choose the 10 coolest ones!!

Silicon Valley has turned back to enterprise IT! For a number of years the buzz was all about the ‘cloud companies’ and everyone wanted to be the next Facebook or Google, but now the money seems to be squarely behind the enterprise now. The venture capitalists have placed their bets and they don’t like to get it wrong!

In the storage space it’s proven to be more interesting than I had predicted! The current ‘battle ground’ is between the traditional way of building storage systems and the use of standardised components with intelligent software. The fundamental attribute of storage is resiliency, after all the purpose of persistent storage is to persist! While the traditional approach of providing ‘no single point of failure’ was through engineering a systems with at least two of everything, the new ‘software resilient’ architecture uses smart software on many standardised systems.

I’m always amazed that we technical people who should be rational and logical, tend to lose perspective in these times of religious fervour! The issue is that the traditional architectures provide far more than just resilience, they are engineered for a specific purpose and thus deliver against a set of service levels. These systems resolve the sharing issues like QOS etc. On the other side of the camp the software developers recognised it has to scale-out from the get-go, needs to be reliable across “not-that-reliable” hardware!

So what is the right way ahead, well I have to say it looks like BOTH ways have their pro’s and con’s and as always in IT, it depends!! So what are we seeing? Firstly traditional workloads tend to gravitate to traditional architectures, and with advances in FLASH and FLASH arrays there are new opportunities and new ways to solve old problems. For example instead of having multiple physical copies of your database, (to separate the loads), today using XtremIO you can run them all on the same array, using the snap copies, and have performance and reliable low latency…to boot!

Then at the other end of the spectrum you have Big Data workloads, where keeping the data close to the CPU pays its dividends in massive processing jobs! So a system like ScaleIO has some interesting applications.

However, in life the ultimate performance may not always be the best solution, when you have to consider all the other aspects of running a production system. Hence systems like Isilon have become very popular in the Big Data space.. a scale-out architecture that implements the Hadoop file system. Here for a small drop in ultimate performance you gain an enterprise class storage system that does not take a team of engineers to maintain etc.

In the end I think the real battleground is how to seamlessly use the right tool for the right job. The challenge is the glue that makes these tools look like one system, with little to no duplication, no increase in management and the automation of the optimal placement of data, at any point of time. Now that’s interesting!

So when your storage vendor arrives with their hammer and tries to convince you that everything is actually a nail. Don’t get religious about it, using the right tool for the right job is always the best option, not only is the TCO always less and performance better, but you get to spend nights and weekends with the family. (And wasn’t that one of your New Year’s resolutions!)

The Year of Big Data – 2014

My New Year’s resolution is to be more consistent with this blog, so please help me as any comments and feedback is motivational!

At this time of the year there are more predictions and forecasts than you poke a stick at, so I’m not going to add to this, but rather delve deeper into one prediction that everyone seems to agree with Big Data. If you have read some of my previous posts you would remember that I believe Big Data is the tools and techniques that help us understand what the data is trying to tell us, and how we use that knowledge.

During the last couple of years I’ve been reading as much as I could on the subject and in this process have become a member of a number of Big Data and Analytics groups. And although I’ve had this kind of interest in a number of different subjects I have never known a group of professionals who are more willing to share their IP. I’m absolutely amazed at how much of the basic work is being developed in the open community.

This willingness to share at almost any level is fuelling this field at an amazing rate. Over the last two years the discussions have mapped progress in this field. Starting with, “What is Big Data?” and “Is there any real practical use?” Through the stages of how to get started and what the team should look like, progressing to about a year ago, where it grew into what were the best tools and flowed naturally on to what algorithms where suited to which jobs.  Now the discussions have become more sophisticated, for example how to use a combination of algorithms which enhance weak signals to provide the best predictions.

(This reminds me my journey learning Photoshop, you learn the basics-opening closing, saving, cropping, white balance, etc. Then you discover the tools that allow you to do things like correcting exposures with levels and curves as well as filters to enhance your image and finally you find people who have worked out how to combine functions to fix difficult problems or do magic to your image; for example the use of the high pass filter to enhance sharpness, or the combination of layers in certain ways to ‘pop’ that sunset!)

So why am I so confident that this is the prediction that everyone might have got right… well it seems to me that we’ve moved into Gartner’s – “Slope of Enlightenment” in Big Data! So much so that I’ve rolled up my sleeves and hit the keyboard over the last few weeks to turn some of my theoretical knowledge into practical experience… which I’ll share here in the coming weeks.