Finding wisdom in a binary world

Sometimes I think that the thirst for data, combined with a digital mindset, is reducing our analytical and critical thinking to a highly polarised, binary-driven view of the world.

Rather than recognizing that most ideas and concepts are composed in “technicolor” we are increasingly reducing our options, choices, responses and decisions to “black or white” conclusions. Everything has to be couched in terms of:

  • on/off
  • yes/no
  • true/false
  • positive/negative
  • for/against
  • like/dislike
  • friend/unfriend
  • connect/disconnect

It feels that our conditioning is driven by the need for certainty, the desire to be “right”, and the tendency to avoid disagreement/difference. However, uncertainty is more prevalent than we may like to admit. To illustrate what I mean, here are four personal learning experiences I would like to share by way of demonstrating that not everything can be reduced to black or white thinking:

1. The scenario is the same, but the context, therefore the answer,  is different

Although I was born and grew up in the UK, I completed part of my primary education in Australia. Before returning to the UK, I was required to complete the 11-plus exam, to determine which secondary school I would attend in England. (The exam was mainly designed to test literacy, numeracy and verbal reasoning.) Here’s a multiple choice question which I got “wrong”:

Q. Why do windows have shutters?

I chose “to keep out the sun” as my answer. In fact, the “correct” answer was “to keep out the wind”. The invigilator was kind enough to include a note to the examiners that my answer was based on the fact that I had been living in Australia, where window shutters are primarily designed to keep out the sun (and therefore the heat). Whereas in the UK, shutters are largely used as a protection against the wind.

2. The facts are the same, but the interpretation is different

While studying for my law degree, I had to write an essay on reforming the use and application of discretionary trusts, based on the current legislation and recent court cases. I argued in favour of an alternative approach to the relevant court decisions, and deliberately took a contrary view based on my social and political outlook at the time, and influenced by what I saw were changes in public policy.

To my great surprise, the tutor gave me one of my highest ever grades in that subject – even though she disagreed with my conclusions, she recognised that my reasoning was sound, and my interpretation was valid.

3. The intention may be “constructive”, but someone will always choose to see only the negative

Early in my career, I participated in a TV documentary series about different types of interview situations. As a local government officer, it was my role to advise members of the public on how to navigate the various regulations and policies in respect to accessing council services, as they related to their own particular circumstances.

One interview I conducted was included in the final broadcast. I thought my advice was objective, and based on widely accepted principles, but without advocating or recommending a specific course of action, as I believed it was my job to remain impartial yet factual. I later discovered that another local council used part of the same interview footage to train their own staff in how not to conduct an interview, because it could have been mis-interpreted as a way to get around the system. So, whereas I thought I was being constructive, someone in a position of authority chose to see it as a negative influence.

4. The assumptions may be reasonable, but the results often prove otherwise

Years later, I found myself having to defend a proposal to launch a smaller, and cheaper, version of a global product in a local market. The received wisdom among many of my colleagues was that the proposal would result in less revenue, even if customer numbers grew. As part of the initiative, I also advocated shutting down a legacy local product in the same market – partly to reduce production costs, and partly because very few customers were actually paying for this outdated service. Again, I faced resistance because a number of internal stakeholders thought customers would refuse to pay for a superior service, and that the business would end up alienating existing customers and, by extension, upsetting the local market.

Subject to a detailed customer migration plan, some very specific financial metrics and frequent status reports, the project was greenlighted. 12 months’ after implementation, the results were:

  • Comparable revenue was doubled
  • Overall production costs were halved
  • A significant number of new clients were signed up (including several from new market segments)

The closure of the legacy product did see the loss of some customers (about 10-15% of the legacy client base), but this was mostly non-paying business, and was more than offset by the increased revenue and customer growth. [In my experience, significant platform migrations and product upgrades can result in up to 20% of customers electing not to switch.]

What are we to conclude from this?

It’s totally understandable that businesses want to deal only with certainty (“just give me the facts…”) and often struggle to accommodate alternative or contrary perspectives. But despite the prevailing digital age of “ones and zeroes”, we are actually operating in a more fluid and diverse environment, where new business opportunities are going to be increasingly less obvious or come from non-traditional sources. While we may find comfort in sticking to core principles, we may end up missing out altogether if we are not prepared to adapt to changing circumstances: context is all about the difference between “data” and “knowledge”.

Wisdom comes from learning to acknowledge (and embrace) ambiguity; individuals, teams, organisations and businesses are more likely to benefit from greater diversity in their thinking, resulting in richer experiences and more beneficial outcomes.

 

How Can I Help?

My purpose in launching this blog was to develop a personal brand, to engage with an audience, and to provide a platform for my ideas and interests, especially in respect to navigating the “information age”.

At the risk of self-aggrandizement, I’d like to think that this blog is helpful, informative and even entertaining. After two years of blogging, I have a sizeable and regular audience, my content gets shared and commented on by numerous readers, and key articles continue to be read many months after publication. (Two of the most popular articles in 2014 were actually published in early 2013.)

Several of my core followers have mentioned why they enjoy my blog, and these are some of their reasons:

  1. The content is original and well written
  2. The articles make them think about things in new ways
  3. I write about novel ideas
  4. My thinking reveals hitherto hidden or less obvious connections
  5. I’m never afraid to state my opinion

Which all suggest to me that they derive value from my analysis and conclusions.

So, my offer of help is this: If you would like access to this creative process, either in support of a specific business opportunity, or to address a strategic issue you face, or simply to help with your own content development, please get in touch via this blog or direct by e-mail. In return, I will provide you with an initial assessment of the issues as I see them, and an outline solution, at no obligation. It’s simply my way of saying “thank you” to everyone who has made an effort to engage with Content in Context.

 

2015 – A Year for Optimism?

After a very challenging 2014, I am trying to face 2015 with a spirit of renewed rationalism and optimism. It won’t be easy, but if we can remain true to our real purpose, and (re)-connect with those things that bring us a sense of joy with the world, maybe we can get through it together. Now, more than ever, we need a Chief Rational Optimist

 

The New Alchemy – Turning #BigData into Valuable Insights

Here’s the paradox facing the consumption and analysis of #BigData: the cost of data collection, storage and distribution may be decreasing, but the effort to turn data into unique, valuable and actionable insights is actually increasing – despite the expanding availability of data mining and visualisation applications.

One colleague has described the deluge of data that businesses are having to deal with as “the firehose of information”. We are almost drowning in data and most of us are navigating up river without a steering implement. At the risk of stretching the aquatic metaphor, it’s rather like the Sorcerer’s Apprentice: we wanted “easy” data, so the internet, mobile devices and social media granted our wish in abundance. But we got lazy/greedy, forgot how to turn the tap off and now we can’t find enough vessels to hold the stuff, let alone figure out what we are going to do with it. Switching analogies, it’s a case of “can’t see the wood for the trees”.

Perhaps it would be helpful to provide some terms of reference: what exactly is “big data”?

First, size definitely matters, especially when you are thinking of investing in new technologies to process more data more often. For any database less than say, 0.5TB, the economies of scale may dissuade you from doing anything other than deploy more processing power and/or capacity, as opposed to paying for a dedicated, super-fast analytics engine. (Of course, the situation also depends on how fast the data is growing, how many transactions or records need to be processed, and how often those records change.)

Second, processing velocity, volume and data variety are also factors – for example, unless you are a major investment bank with a need for high-frequency, low-latency algorithmic market trading solutions, then you can probably make do with off-the-shelf order routing and processing platforms. Even “near real-time” data processing speeds may be overkill for what you are trying to analyze. Here’s a case in point:

Slick advertorial content, and I agree that the insights (and opportunities) are in the delta – what’s changed, what’s different? But do I really need to know what my customers are doing every 15 seconds? For a start, it might have been helpful to explain what APM is (I had to Google it, and CA did not come up in the Top 10 results). Then explain what it is about the resulting analytics that NAB is now using to drive business results. For instance, what does it really mean if peak mobile banking usage is 8-9am (and did I really need an APM solution to find this out?) Are NAB going to lease more mobile bandwidth to support client access on commuter trains? Has NAB considered push technology to give clients account balances at scheduled times? Is NAB adopting technology to shape transactional and service pricing according to peak demand? (Note: when discussing this example with some colleagues, we found it ironic that a simple inter-bank transfer can still take several days before the money reaches your account…)

Third, there are trade-offs when dealing with structured versus non-structured data. Buying dedicated analytics engines may make sense when you want to do deep mining of structured data (“tell me what I already know about my customers”), but that might only work if the data resides in a single location, or in multiple sites that can easily communicate with each other. Often, highly structured data is also highly siloed, meaning the efficiency gains may be marginal unless the analytics engine can do the data trawling and transformation more effectively than traditional data interrogation (e.g., query and matching tools). On the other hand, the real value may be in unstructured data (“tell me something about my customers I don’t know”), typically captured in a single location but usually monitored only for visitor volume or stickiness (e.g., a customer feedback portal or user bulletin board).

So, to data visualisation.

Put simplistically, if a picture can paint a thousand words, data visualisation should be able to unearth the nuggets of gold sitting in your data warehouse. Our “visual language” is capable of identifying patterns as well as discerning abstract forms, of describing subtle nuances of shade as well as defining stark tonal contrasts. But I think we are still working towards a visual taxonomy that can turn data into meaningful and actionable insights. A good example of this might be so-called sentiment analysis (e.g., derived from social media commentary), where content can be weighted and scored (positive/negative, frequency, number of followers, level of sharing, influence ranking) to show what your customers might be saying about your brand on Twitter or Facebook. The resulting heat map may reveal what topics are hot, but unless you can establish some benchmarks, or distinguish between genuine customers and “followers for hire”, or can identify other connections with this data (e.g., links with your CRM system), it’s an interesting abstract image but can you really understand what it is saying?

Another area where data visualisation is being used is in targeted marketing based on customer profiles and sales history (e.g., location-based promotion using NFC solutions powered by data analytics). For example, with more self-serve check-outs, supermarkets have to re-think where they place the impulse-buy confectionary displays (and those magazine racks that were great for killing time while queuing up to pay…). What if they could scan your shopping items as you place them in your basket, and combined with what they already know about your shopping habits, they could map your journey around the store to predict what’s on your shopping list, thereby prompting you via your smart phone (or the basket itself?) towards your regular items, even saving you time in the process. And then they reward you with a special “in-store only” offer on your favourite chocolate. Sounds a bit spooky, but we know retailers already do something similar with their existing loyalty cards and reward programs.

Finally, what are some of the tools that businesses are using? Here are just a few that I have heard mentioned recently (please note I have not used any of these myself, although I have seen sales demos of some applications – these are definitely not personal recommendations, and you should obviously do your own research and due diligence):

For managing and distributing big data, Apache Hadoop was name-checked at a financial data conference I attended last month, along with kdb+ to process large time-series data, and GetGo to power faster download speeds. Python was cited for developing machine learning and even predictive tools, while DataWatch is taking its data transformation platform into real-time social media sentiment analysis (including heat and field map visualisation). YellowFin is an established dashboard reporting tool for BI analytics and monitoring, and of course Tableau is a popular visualisation solution for multiple data types. Lastly, ThoughtWeb combines deep data mining (e.g., finding hitherto unknown connections between people, businesses and projects via media coverage, social networks and company filings) with innovative visualisation and data display.

Next week: a few profundities (and many expletives) from Dave McClure of 500 Startups