Information-Driven Business
How to Manage Data and Information for Maximum Advantage

Robert Hillard
home > blog > post

Your insight might protect your job
by Robert Hillard

Technology can make us lazy.  In the 1970s and 80s we worried that the calculator would rob kids of insight into the mathematics they were learning.  There has long been evidence that writing long-hand and reading from paper are far superior vehicles for absorbing knowledge than typing and reading from a screen.  Now we need to wonder whether that ultimate pinnacle of humanity’s knowledge, the internet, is actually a negative for businesses and government.

The internet has made a world of experience available to anyone who is willing to spend a few minutes seeking out the connections.  Increasingly we are using big data analytics to pull this knowledge together in an automated way.  Either way, the summed mass of human knowledge often appears to speak as one voice rather than the cacophony that you might expect of a crowd.

Is the crowd killing brilliance?

The crowd quickly sorts out the right answer from the wrong when there is a clear point of reference.  The crowd is really good at responding to even complex questions.  The more black or white the answer is, the better the crowd is at coming to a conclusion.  Even creative services, such as website design, are still problems with a right or wrong answer (even if there is more than one) and are well suited to crowd sourcing.

As the interpretation of the question or weighting of the answer becomes more subjective, it becomes harder to discern the direction that the crowd is pointing with certainty.  The lone voice with a dissenting, but insightful, opinion can be shouted down by the mob.

The power of the internet to answer questions is being used to test new business ideas just as quickly as to find out the population of Nicaragua.  Everything from credit cards to consumer devices are being iteratively crowd sourced and crowd tested to great effect.  Rather than losing months to focus groups, product design and marketing, smart companies are asking their customers what they want, getting them involved in building it and then getting early adopters to provide almost instant feedback.

However, the positive can quickly turn negative.  The crowd comments early and often.  The consensus usually reinforces the dominant view.  Like a bad reality show, great ideas are voted off before they have a chance to prove themselves.  If the idea is too left-field and doesn’t fit a known need, the crowd often doesn’t understand the opportunity.

Automating the crowd

In the 1960s and 1970s, many scientists argued that an artificial brain would display true intelligence within the bounds of the twentieth century.  Research efforts largely ground to a halt as approach after approach turned out to be a dead-end.

Many now argue that twenty-first century analytics is bridging the gap.  By understanding what the crowd has said and finding the response to millions, hundreds of millions and even billions of similar scenarios the machine is able to provide a sensible response.  This approach even shows promise of meeting the famous Turning test.

While many argue that big data analytics is the foundation of artificial intelligence, it isn’t providing the basis of brilliant or creative insight.  IBM’s Watson might be able to perform amazing feats in games of Jeopardy but the machine is still only regurgitating the wisdom of the crowd in the form of millions of answers that have been accumulated on the internet.

No amount of the crowd or analytics can yet make a major creative leap.  This is arguably the boundary of analytics in the search for artificial intelligence.

Digital Disruption could take out white collar jobs

For the first time digital disruption, using big data analytics, is putting white collar jobs at the same risk of automation that blue collar worker have had to navigate over the last fifty years.  Previously we assumed process automation would solve everything, but our organisations have become far too complex.

Business process management or automation has reached a natural limit in taking out clerical workers.  As processes have become more complex, and their number of interactions has grown exponentially, it has become normal for the majority of instances to display some sort of exception.  Employees have gone from running processes to handling exceptions.  The change in job function has largely masked the loss of traditional clerical works since the start of mass rollout of business IT.

Most of this exception handling, though, requires insight but no intuitive leap.  When asked, employees will tell you that their skill is to know how to connect the dots in a standard way to every unique circumstance.

Within organisations, email and, increasingly, social platforms have been the tools of choice for collaboration and crowdsourcing solutions to individual process exceptions.  Just as big data analytics is automating the hunt for answers on the internet, it is now starting to offer the promise of the same automation within the enterprise.

In the near future, applications driven by big data analytics will allow computers to move from automating processes to also handling any exceptions in a way that will feel almost human to customers of everything from bank mortgages to electric utilities.

Where to next for the jobs?

Just as many white collar jobs have moved from running processes in the 70s and 80s to handling their exceptions in the 90s and new millennium, these same jobs need to move now to find something new.

At the same time, the businesses they work for are being disrupted by the same digital forces and are looking for new sources of revenue.

These two drivers may come together to offer an opportunity for those who spent their time handling exceptions either for customer or internal processes.  Future opportunities are in spotting opportunities in business through intuitive insights and creative leaps and turning them into product or service inventions rather than seeking permission from the crowd who will force a return to the conservative norm.

Perhaps this is why design thinking and similar creative approaches to business have suddenly joined the mainstream.

comments powered by Disqus


blogThe Information-Driven Business blog is published monthly:

2017   2016   2015   2014   2013   2012   2011   2010


Also featured from the Information-Driven Business blog:

Who are we leaving behind?
I was recently invited to deliver the keynote address at the University of Melbourne engineering and IT awards night. I took the opportunity to challenge today’s students to think about the people being left behind as in the move to … Continue reading

The Quantum Computer dream could be killed by Information Management
For years now the physics community has been taking the leap into computer science through the pursuit of the quantum computer.  As weird as the concepts underpinning the idea of such a device are, even weirder is the threat that … Continue reading

Customers smile at the edges
One of the best ways of understanding your success is by measuring how happy your customers are.  To make sense of their response you obviously need to segment customers.  In this post I propose segmenting customers according to the complexity … Continue reading




© 2010-2017 Robert Hillard | Privacy Policy