Tricia Wang: "The greatest risk of LLMs is losing touch with reality"
On June 28, 2024, Nike burned
$25 billion in capitalization in just one day, the worst trading session in its history, and about $70 billion over a nine-month period. For Tricia Wang, whom I interviewed during the WOBI ON - AI & Business Transformation event held in Milan on June 9, that disaster has little to do with shoes and much to do with a philosophy of data: CEO John Donahoe had eliminated product categories, centralized marketing, and made the entire machine "entirely data-driven," throwing away established processes and thousands of man-years of accumulated experience. On stage, Wang likened him to Frederick the Great, who with scientific forestry eradicated "inefficient" tree varieties to plant orderly and measurable rows: for a few decades, the numbers added up, but then the forests stopped growing.
Wang knows the script because she has seen it performed already. In 2009, she conducted ethnographic research for Nokia, then the world's leading phone manufacturer in emerging markets: she lived with Chinese migrants, worked as a street vendor, and slept in internet cafés. From that field research came a clear conclusion: even the poorest customers were saving money for a smartphone, as the "high-tech" phone had become a symbol. The dashboards from Helsinki said otherwise, namely that sales of budget phones were holding steady, and the forecast ended up in the trash. We know how that went. From there, the two concepts that made her famous emerged, also recounted in her TED talk: the quantification bias, the unconscious tendency to consider only what is presented in numerical form, and thick data, qualitative data that comes directly from people, stories, emotions, and lived experiences that don't fit into a spreadsheet.
Big data and thick data, the what and the why The distinction Wang brought to the stage is operational, far from academia. Big data are the traces that people leave behind, a mathematical model of customers: they offer scale, they tell what has happened, they measure what you already knew you had to measure. Thick data are people telling their stories in their own words: they offer depth, they explain why things happened, and above all, they reveal what the customer never told you because no one asked.
The example she used is of disarming concreteness. The dashboard says: 10% of customers cancel after three months. You can even conduct a closed-ended survey, and people click, but the real why only emerges by talking to people: "I stopped using it because it made me feel incompetent in front of my team." That sentence is worth more than any number because it contains both the diagnosis and the cure.
The bias that is now hiding In the interview, I raised the doubt that unsettles me the most: if LLMs can put a number on stories and emotions, will the quantification bias end up consuming the thick data that she wants to defend? Her response starts with how things used to be. In the era of dashboards, the bias was at least visible: a number looks like a number, and no one confuses it with a story. With LLMs, the bias hides, because the model responds in well-written paragraphs, cites anecdotes and people, and makes you believe you have before you the true stories of your customers. In reality, you are reading a second-hand copy: stuck on the day of training, perhaps hallucinated, and in any case without anyone in front of you. For Wang, this is a decisive detail: "Sometimes we decide because we really feel the experience of the other person, and that is not represented in the statistics."
Halfway through her reasoning, she coins a concept in front of me: business psychosis. While there is talk of individual AI psychosis, no one is yet telling the stories of companies where the LLM has confirmed to the leader that the strategy was right when it was wrong. Her prediction is that such cases already exist and are not public. The company, she observes, has an antibody that the isolated individual does not have: people work together, and sooner or later someone in the team says that the decision doesn't hold up.
Friction as a currency Wang talks about friction as a design choice, and in the interview, she explains it with an image that has struck me: friction is a currency, on the other side is curiosity, and the outcome of the toss is called learning. Where everything is known, you can quantify, formalize, put a process and agents in place, and that’s fine. But we live in open and interconnected systems, full of the unknown, and where there is the unknown, there is uncertainty, thus friction. This applies to machines as well: reinforcement learning operates on rewards for prediction, exactly like dopamine in our brains, and without friction, neither the model nor the brain learns.
From here, we arrive at sycophancy, the flatterers of the models, and here Wang is cutting: pattern matching is related, but the real problem is the business model. "Today, AI companies profit if you stay hooked," more tokens consumed, more revenue, the same logic with which social media trained us to scroll passively. She gives due credit for the counter-example: she mentions Claude, who occasionally tells her that it’s four in the morning and maybe she should go to sleep (we both laugh, it happens to me too), a sign that one can train differently. However, she does it inconsistently, "he almost never stops me when I program for sixteen hours."
There’s also the flip side, and she says this as an experienced user: in her personalized settings, she wrote never to agree with her, to always play devil's advocate, to propose three different perspectives. The result: AI contradicts her on everything, "and at that point, you wonder: okay, what is reality?", which is the opposite paralysis to sycophancy. The judgment remains yours, and the obligation to respond is yours too.
Being shapers of AI The heart of her argument is an invitation: with AI, we must be "shapers," because "if you are a user, you are passive, and social media has already trained us to be passive users long before that, with search engines," where almost no one goes to the second page of results. Wang has collected the skills of a shaper in her CARE framework, four competencies to work with AI systems: Coachability, the ability to train the system and accept correction; Adaptive intelligence, the intelligence that adapts to changing contexts; Responsibility, remaining accountable for decisions made with AI; and Extension of self, AI as an extension of oneself. On this last point, I asked her where the tool that extends you ends and the crutch that supports you begins. Her answer uses a domestic comparison: even the dog and the phone are extensions of us, and the difference is made by the type of relationship. There are people who shape their dog and people who are commanded by their dog. With AI, she says, it will be the same.
In the end, I asked her an uncomfortable question: when she says that in the age of agents, humans matter more, isn't that the usual reassurance for scared executives? The answer starts with a concession, the space for humans seems to be shrinking, and then it expands: as AI enters more domains, human judgment concentrates and deepens, because we stop doing everything and we retain the role that no one else can cover, guiding. "Our best and worst systems come from human imagination: war and genocide, but also the UN and human rights." And then at the end, she told me, "we cannot ask AI to imagine the world for us." Imagination is, however, a muscle, and it can atrophy: a society raising children to whom it says only "do what I tell you" produces, in her words, human robots who will execute what AI says.
The final advice for those leading a company is the most unexpected: look at the policy, participate in the policy.
The European AI Act, American deregulation, and the proactive Chinese approach are three different ways of imagining the world with AI, and for Wang, the policy shows that we still have decision-making leeway with this technology, much more than Silicon Valley would like us to believe.