Are you ready to get involved in the Human Face of Big Data?

Starting next week the HFoBD project will tempt just 10 million of you to download the app and begin taking part in the largest data collection project in the history of mankind. Rick Smolen’s vision to create a digital mirror on the world is about to go live. (Short clip of Rick talking about the project, here)

You can feel a large affinity to this project as Rick’s rise to fame has its genesis right here in Australia, where he and 100 of his photo-journalist mates, set out to capture a ‘Day in the life of Australia’. A creative and crazy idea that no publisher would back, hence his production company name, “Against All Odds”.

(How to get involved? Simply register at www.thehumanfaceofbigdata.com, and/or download the app on the 25/26^th from the Apple or Droid store.)

Think about getting 10Million people, (maybe more), answering a bunch of questions and to keep them engaged feeding back information and fun facts! Now do the maths and work out how much data needs to be flowing around the globe! If each second each user accounts for just 10 bytes of data, then we are talking 100Mbytes per second, or for you network guys .. in the order of a GigaBit!

Now Rick has spoken about finding your ‘Digital Doubleganger”, where your answers will be matched with others and you can share a picture of yourself with your digital twin. So now we are talking about pictures, which will be a little more than 10 bytes.

An interesting challenge when it comes to the infrastructure, especially when you consider that on the 2^nd October we will have three Mission Control’s, (Singapore, London and NY).  Here press and analysts will be able to view and interact with this project, using some fascinating work that Tablaeau is doing to be able to visualise the data collected.

So keep posted and become involved.

Be a Part of Time Travel – The Human Face of Big Data

Imagine if you could have interacted with today’s www fifteen years ago. I believe that ‘Big Data’ is going to have an even more profound effect than the internet has had on our lives, our work and even our play! The Human Face of Big Data is a project that is designed to give us a glimpse into this future. (www.thehumanfaceofbigdata.com)

To me this thing that is being called ‘Big Data’ is quite simply a shift in what we use technology for. In the past technology has in general automated what we could already do manually. We could keep customer records, write out purchase orders, hand write notes, even walk over and talk to people! (OK that last one was a lot harder if they lived on the other side of the world!)

The current transition is where we use technology to supplement our brains, which it is said can consider 5, (give or take 2), things at one time. We have trained ourselves to cope with this by summarising things down, focusing in on what we believe are the important pieces etc. However, there is a whole new approach to being “inclusive” in our thinking rather than “exclusive”.

Einstein once explained that the difference between people like him and the rest of us is that if he was asked to find a needle in a haystack, he would continue to look for all the possible needles, rather than what we would do is to stop after finding the first. (Interesting article on ‘inclusive’ thinking here.)

You can be part of this social experiment! The project involves gathering data from TEN MILLION people around the world. Data about their everyday lives which will create a “Big Data” set to help demonstrate the power of data. Just go to www.thehumanfaceofbigdata.com and register or just download the app on the 25^th September and begin providing data to this project.

Finding your way in Big Data.

Do you use maps in your organisation today? Like me, I would guess most of you would say, no!

Do you use maps outside work? Like me, I would say many times every day…

Drive to work, just check the traffic!  Going to a new place, just start the GPS!  Want a restaurant, lets Google/Uban Spoon/etc.!  Need petrol, where is the nearest?  Wonder where my son is, let me look at ‘find my friends’.  Going on holiday?

So why when we use on-line interactive and mashed up maps everyday personally do we not build these capabilities into our systems? Last week listening to the people from Esri I realised just how powerful mapping is to big data.

I spent last week with the folks of Esri at their annual conference Ozri, (Esri Australia is the nation’s leading Geographic Information System (GIS) specialist here).  They prove the tools and systems you need to operationalize maps.

Now we are all familiar with the map-mashup, we use it everyday where information is overlayed on the map. (If you go back in my blog about a year, (here), you see my cycling application overlaying GPS data on the map… so I can see terrain, segments, etc.   But there were so many new use cases I was not aware of…

3D maps are fascinating and provide a whole new dimension to their applicability, (dad joke).  When you think 3D don’t think ground maps and don’t think static. Firstly think mines, construction sites, city plans, etc and then think ‘modelling and simulation’ on top of these, then think visualisation!  Now walk through a new city, and change planning assumptions as you go. Build the new railway line and change zoning along it to see what happens. Build a mine and visualise what happens as different assumptions are made.

Why stop there, add the fourth dimension time, now watch satellite imagery of the third runway being built, (Sydney Airport), and the resultant change in vegetation on the adjacent shores.  Calculate how cities are encroaching on surrounding farm-land and extrapolate it out 50 years.

The last real sexy demonstration I’ll talk about is their integration into Microsoft Office! Plonk a map into your spread sheet and ‘feed’ rows into the map, (which contain location information). Immediately pick up patterns, hot-spots, even filter the data and watch how things change.

Now try and think about what your organisation does and how location can add a new dimension! As someone said at the conference, GIS adds context to data and remember that everything that happens, happens somewhere. GIS the gateway to Big Data!

What is Software Defined Networking and why is it heating up?

Over the past couple of months a few start-up SDN companies have been purchased by major vendors, e.g. VMware picking up Nicra. Do you need to take note and what’s going on?

Firstly YES, you need to be on-top of this as it will affect your future! The issue here is that traditional networking is causing a number of issues as it is too restrictive, to take us into the future of the “hybrid cloud”. So in summary, in the future you will still have your physical topology as well as essentially a logical one, the logical one is basically where SDN comes into play.

To simplify the problem right down the IP system was designed to connect two devices together. Two issues with this as we move to the cloud are:

–       In its design, it starts by finding a path between the two devices, and then connects them together. Now when one of the devices suddenly moves, the system breaks down!

–       The interface today describes the devices identity, (my MAC address is me!), however today we are more interested in the service being provided. Similar to the change in virtualising servers, we used to think about a physical box being our server, (“There is our Exchange server!”), now we think about the application,(“Which VMware cluster is your Exchange running on?”), not related to a specific box.

So there is a need to separate out the control of the network traffic and retain the physical networking to move the data in-order to move into a dynamic and service oriented world.

There are a couple of major initiatives underway in this space, such as the ‘Open Network Foundation (ONF) founded last year by a consortium of vendors and end users to promote the use of SDN. Their major piece of work is to create an open interface standard which is called ‘Openflow.’ As expected, enterprise networking giant Cisco is also working in this area and they have an API/SDK kit called the ‘Open Network Environment Platform Kit, (OnePK), to foster an ecosystem around their switching and routing devices.

To try bring this all together, imagine you are a cloud provider. You need to deliver to your customers a network which looks just like what they have on their site and you need to deliver the service level that you are contracted to them to deliver, however you have to provision this from your highly leveraged infrastructure. How do you do that? You put a ‘virtual network layer’ on top of your physical one, let’s call that a software defined network!