Analytics is Subjective

...or at least can never be "purely" objective.

Any decision relating to risk involves two distinct and yet inseparable elements: the objective facts and a subjective view about the desirability of what is gained, or lost, by the decision. Both objective measurement and subjective degrees of belief are essential; neither is sufficient by itself.

Against the Gods: The Remarkable Story of Risk - 1998

This quote describes that there are two things going on in any decision in the face of risk:

1. How likely is any given outcome.
2. How good or bad would it be if any of the outcomes occur.

The first one is where analytics is often positioned. This is the objective world of simulation, probability and game theory.

The second is what I'd like to talk about here, because while some of it is hard nosed scenario analysis, there is another element to it. Maths and simulation can have a reasonable chance at predicting the knock-on impacts of an outcome - but just like the quote implies - the question of whether those impacts are "good" or "bad" is inherently... subjective.

Why?

When making assumptions it is only natural that among a group of people, there will be different beliefs on the strength of each assumption which is used as an input. However on top of that, we are trying to assess the implications of a decision on a complex system. Complex systems are almost by definition non-deterministic. Subtle differences in beliefs, assumptions and information might tip the scales in approximating the behavior of that system and so steer the subjective view of the implications of a given outcome. Each human mind is trying to estimate the implication of the outcome on a complex system using a bundle of learned heuristics and past experience. I argue that any practical situation a professional analyst is working on is very unlikely to behave deterministically enough for this effect to not be meaningful. Customer buying behavior, animal nutrition & digestion, the voting decisions of an electorate, the morale & opinions of a large organisation - are all complex systems.

So what?

We view the role of an analyst sometimes to "tell us the facts" or "help us see what the numbers are telling us" - but this is only half the story. To make meaningful decisions (or even helpful recommendations) in the face of any practical decision - even if we can be totally objective on the probability of an outcome - we must take a subjective view on the implications of that outcome.

But if it's subjective what role can an analyst play?

The core skills of an analyst are not objectivity and an invulnerability to bias, I would argue they are the practical structuring of complex problems and the knack to identify the right question to ask. Just because it is subjective, does not mean that these two skills are not still a way to make progress - just that the application of them is different.

The application of structure is very similar, regardless of how subjective the question. However if in objective measurement and analysis we must collect objective facts and data - then in subjective analysis we must collect subjective data.

Subjective facts?

Surely facts are objective, quite the opposite of subjective? A subjective fact, would not be the fact itself, but instead would be the perceptions and beliefs wrapped up around that fact. The different hopes & fears each person associates with a given outcome or objective situation. This is precisely what an analyst must seek out to understand the subjective implication, and therefore really get a handle on risk. Typically we ask questions to cut away the subjectivity, especially when trying to first get to objective reality. In is case however we must do the opposite, because we are seeking what we normally choose to leave on the cutting room floor. This means that we need to ask different kinds of questions - in an effort to get a different kinds of answer.

This is where I believe we as data professionals so often fail. When we are viewed as "too objective" or "cold" or more often simply can't get groups of people to agree on a common path forward when we feel the objective facts are clear. It's so easy the get frustrated and claim that "they're not listening". The truth is so often the opposite - that we are not listening, or worse that we never asked.

If you feel the facts are clear put that a group can't reach agreement - it's likely because while the objective facts are clear, the subjective facts are not. Step back, structure the problem and start seeking the subjective views of the people in the room. Ask about not what people think, but what they feel, and analyse that.
Given two people where one has a fear of flying: the choice that one takes not to fly may be right for them, and not for the other. Anyone who has tried to reassure someone who is afraid of flying by quoting the objective relative probability of death in a flight as compared to driving, knows that a better understanding of objective reality is not going to help the situation. This is because the difference in their fears is subjective and comes from differing views on the implications of a plane crash - not because of a differing brief on the likelihood of a crash.
In this case it is irrelevant whether that subjective view is valid in our opinion* - if we don't understand it and take it into account in our actions we will be ineffective, or worse: downright unhelpful.

The same is true in our day jobs. 😁

* IMHO I think in most businesses setting, this difference in subjective belief is actually a good thing on most problems, but that's probably the subject of a whole separate post.