The other day, I noticed a very interesting discussion unfolding regarding a product problem within our company's group chat. I won't reveal too much about the nature of the issue because I don't want to lose my job 😛.
This discussion ping-ponged back and forth with opposing viewpoints on the problem, and no clear insights emerged. In fact, even after several days of discussion, it seemed as though the root cause hadn't been discovered, and conflicting opinions still prevailed. It's worth noting that all of the discussion points were heavily backed by data.
Just imagine a thread with people hurling charts and numbers at each other, debating opposing viewpoints. This is data analyst heaven - people arguing using data-backed insights instead of random opinions!
At one point, I noticed one of the senior product managers stepping in and using the phrase "Analysis Paralysis". I'd never actually heard of this term before because no employees in other companies I'd been in had used data to argue like this.
Scale and knowledge silos
As most analysts or data professionals are accustomed to seeing decisions made without any data, the aforementioned situation may seem unusual, but it does occur.
As companies scale and the amount of available data grows concurrently, data teams can become siloed. No data analyst at Google, for instance, can possibly understand or query every area of the business. This results in knowledge silos where comprehension of certain business areas is restricted to a small group of people. Couple this with interlinked products (where changes in one area can affect another), competition, macroeconomic factors, and general consumer behavior changes, and you have a complex mix of variables that can be analyzed in many different ways.
Most of these analyses may not present the complete picture - each business area might provide different explanations for problems. For example, if Instagram’s monthly active users (MAUs) decline, the explanations might vary:
Reels' analysts might attribute the decrease to lower retention rates for their product.
Stories’ analysts might suggest it’s due to a product change on the homepage, leading to fewer displayed stories.
Engineering might argue that slower loading times due to server issues are the culprit.
Threads analysts (for those unaware, Threads is their new Twitter substitute) might find this surprising, claiming that the launch of the Threads product increased activity.
Big questions that affect multiple business areas can be explained in various ways. In the above scenario, Instagram’s drop in MAUs might be due to a combination of the factors described. Understanding how each area impacts another is the responsibility of senior decision makers. These decision makers, often a mix of executives and lead analysts, must consolidate all the information provided to formulate a reasonable hypothesis.
The issue arises when these senior decision makers become fixated on identifying the root cause and consequently spiral into an endless analytics spiral. Question after question might be directed at the team to uncover the issue. This isn't necessarily a bad thing - in fact, I wish more companies operated this way. However, not every problem can be easily unraveled like this. Sometimes, even big data might not be sufficient.
With opposing viewpoints and different data points being used to justify them, it's quite easy to fall into decision paralysis. The senior decision makers may find themselves uncertain of the answer and unsure of what to do.
As a data professional, I hesitate to say this, but it can be helpful to step outside the data in such situations. Consider conducting a few consumer interviews, addressing the problems that have already been identified, inviting fresh perspectives, or even setting the problem aside for a while and returning for further analysis in the future with a fresh mindset.
The future of this newsletter
Unfortunately, I haven't had much time to delve deeply into topics like the one in the following article, which I highly recommend:
I plan to experiment with shorter posts for a few weeks and monitor engagement levels. Additionally, I thought it would be beneficial to get feedback from my readers about this new approach. If you're new here, please remember that my articles will primarily focus on data and tech, with occasional social commentary.