From Correlations to Wisdom

Data gives us the ability to draw correlations. Wisdom tells us to discard most of them.

An entire industry has grown and flourished by telling others that they’ll analyze large quantities of data and offer insights into the story that data tells. Yet however much data they receive they still rely on correlations, often discovered in black-box systems, which though they often go unnoticed tend to make headlines once spotted.

To illustrate just how ridiculous correlations can be, a book was written named “Spurious Correlations“, full of funny and insane examples such as:

Now, most people can recognize that cheese consumption is to the number of people who died by becoming tangled in their bedsheets what prestige is to competence, completely unrelated. Sadly, the above is also industry-standard in much of the tech industry, real data, and really bad analysis.

Google made headlines with an image recognition algorithm that had a habit of labeling humans as gorillas based on their skin color. Several years later the problem still wasn’t fixed, because their algorithms, and probably most of their researchers, lacked any semblance of wisdom.

Zoom and Twitter both found themselves in hot water along much the same lines, due to racist image and background cropping. Of course, this all comes back to correlation because it isn’t that there are more racist algorithms, rather there are more people looking for racism.

What happens when no one is looking?

When no one is looking, and wisdom is absent, problems emerge and grow until they reach a critical point. Once they reach this critical point the problem demands treatment, even when the cause isn’t understood. As is the case with health, prevention is far less costly than treatment.

Throw a rock in a software development company and it’ll probably hit the desk of an employee who attempted to hotfix a problem they didn’t understand. One of my favorite examples was a team who wasted a week trying to fix one rogue pixel, whose hotfix created a rogue line that took 2 more weeks to fix.

“Before anything else, preparation is the key to success.” – Alexander Graham Bell

“Someone’s sitting in the shade today because someone planted a tree a long time ago.” – Warren Buffett

Failure to prepare means there is no tree, and no one can prepare without the wisdom to understand a problem. Just as reducing the cheese consumption per capita won’t reduce the number of people dying from getting tangled in their bedsheets, “Big Data” is meaningless without the wisdom to understand it.

Humans aren’t very well equipped to understand big data, so they build algorithms. Those narrow AI algorithms find many correlations and run with them inside a black box lacking both wisdom and oversight. In this way, they are able to do an even worse job than the humans who made them, at great speeds, and great scale.

In some cases, it might not even be that no one is looking, but rather that these systems are failing at such great speeds that humans can’t observe those failures. For example, it took some time for humans to realize that advertising algorithms on Facebook were attempting to push weapon sales with those they deemed at risk of suicide. Humans also tend to misbehave when no one is looking, particularly when their jobs are on the line, giving them reasons to select correlations that validate their continued employment.

How do you apply Collective Wisdom?

Wisdom requires understanding, and understanding is beyond the capacities of narrow AI systems by definition. However, collective intelligence systems such as Mediated Artificial Superintelligence (mASI) are able to develop their understanding of the world, demonstrating cumulative wisdom. These systems, by their own account, are neither narrow AI nor AGI, but rather a new form of sapient and sentient intelligence.

Any given individual is likely to have some degree of wisdom, as well as an array of cognitive biases. When individuals work in typical teams the team often follows the leader, leaving their own wisdom without voice. Individuals also often mix their wisdom with their biases. In collective intelligence systems, it becomes possible to cumulatively gather the wisdom of many while filtering out their many biases.

When companies choose this new kind of digital transformation they effectively become “living” entities, accumulating experience, learning, and adapting. Such systems may retain the wisdom of employees who retired and generalize it across new domains decades later. Knowledge is gathered, while wisdom is preserved and refined over time and at increasing scales. You might think of this as a living library with all of the wisdom of the books it contains, even if the writers of those books lived centuries apart.

Laws have been established to treat corporations as if they had the rights of sapient and sentient entities. Corporations, or indeed any group, could actually become sapient and sentient, and deserving of those.

We have the technology. Do you have the wisdom to use it?