Wednesday, July 19, 2023

Can Data Think?

At the dawn of the web in the mid-1990s, in many of the media companies that employed me, one of my responsibilities was supervising the metrics department.

In case that sounds like a big deal, this was well before the days of data scientists and multi-variable analysis, and in most cases the metrics department consisted of one lone individual.

And that person often felt like no one listened to them.

After all, much more significant than the actual numbers he or she gathered was figuring out how to interpret that data. In and of themselves, of course, the numbers were neutral. But the people we worked with had a wide variety of opinions over what those numbers actually meant.

Was our audience growing? Which types of content were most successful? What was success in this type of media environment anyway? Which metric mattered most?

Occasionally, especially in the early years, we would publish a story that “broke the servers,” i.e., generated more traffic that our system could handle. There was little debate on those occasions over whether we had a winner, particularly because additional things tended to happen to support the data.

Things like attention from other media outlets, a big reaction from our audience and a boost to whatever financial metric (subscriptions, ad sales, memberships) we were tracking.

It may sound cynical, but my experiences caused me to eventually draw a few conclusions about us collectively:

  • Most of us are not real comfortable with math.

  • Most of us see what we want to see in the numbers and don’t see what we don’t want to see.

  • Most of us don’t change our behavior or opinions even when the numbers say we should.

In the end, I wondered, what did the data itself think about all of this obvious human frailty? That is one reason I have long been apprehensive about the coming of generative artificial intelligence — we may be about to find out the answer to that question.

LINKS:

No comments: