When Algorithms Learn Your Price: The Quiet Rise of Digital Feudalism

As US regulators weigh new rules on personalized pricing, a deeper question emerges: what happens when businesses use AI to learn the exact financial breaking point of every worker and consumer? The answer, according to scholars at the University of Auckland Business School, points toward a system where visibility flows only one way, and the gains from efficiency are captured by those who already hold power.

The Federal Trade Commission is currently consulting on an enforcement policy for personalized pricing, a practice that uses personal data to tailor prices and discounts to individual customers. Meanwhile, Consumer NZ has warned about the vast troves of data collected through supermarket loyalty programmes, data that could reveal how much each shopper is prepared to pay.

Who benefits when AI knows your limits?

Consider the same person in two markets. As a worker, their employer benefits from knowing the lowest wage they will accept. As a customer, a seller benefits from knowing the highest price they will pay. Traditionally, neither side knew those numbers precisely. A worker might accept $24 but receive $30 because that is the going rate. A customer might pay $20 but buy for $14 because that is the advertised price.

Algorithms are rapidly reducing that uncertainty, and the asymmetry is stark. Digital platforms can observe thousands of individual decisions. A ride-hailing platform sees which jobs a driver accepts, when they work, and which incentives bring them online. A retailer sees purchases, abandoned carts, and responses to discounts. The worker and the consumer, meanwhile, remain largely in the dark about the systems governing their transactions.

At its extreme, this risks becoming a kind of digital feudalism: platforms can increasingly see the people they deal with, while those people can barely see the systems governing the exchange.

Lyft has already documented systems that determine which drivers receive incentives, with some earnings challenges explicitly personalized. Research on 1.5 million Uber trips in the UK found dynamic pricing was associated with lower real hourly earnings and greater inequality. A recent FTC investigation found pricing intermediaries had access to location, demographics, browsing histories, shopping-cart activity, and even mouse movements.

How personalized pricing deepens systemic inequality

Markets have never been perfectly transparent. Employers have always known more about wage structures than workers, and sellers more about margins than buyers. Yet there has traditionally been uncertainty on both sides. Algorithmic systems now risk reducing that uncertainty in only one direction.

A worker cannot easily know whether rejecting $24 would have produced $28. Nor can a customer know whether walking away from a purchase today would have triggered a discount tomorrow. Meanwhile, firms can observe, test, and learn from repeated behaviour. The result is a structural shift in bargaining power, one that disproportionately impacts those already marginalized by the extractive logic of late capitalism.

Is AI-driven personalisation ever ethical?

There can be genuine benefits. Targeted incentives can improve matching, personalised discounts can help price-sensitive customers, and better forecasting can reduce waste. But the issue is not whether these systems can create efficiencies. The question is how the gains are distributed.

Personal data has economic value because it can help predict the terms people are willing to accept, making privacy a question of bargaining power too. Workers and consumers are increasingly visible to businesses, while the systems making decisions about them remain largely opaque. They might reasonably expect to know when an offer has been personalised, what information influenced it, and whether others are receiving materially different treatment.

What role should business schools play?

Business schools carry a responsibility here. Alongside teaching pricing strategy, segmentation, and AI-driven decision-making, students should be encouraged to ask: effective for whom? There is a difference between using technology to create new value and becoming better at capturing value from the other side of a transaction.

The most troubling outcome does not require malicious AI. Companies can rationally reduce costs and improve margins while becoming better at predicting what workers will accept and customers will pay. The question cannot simply be whether something can be optimised. We should also ask who benefits, whether it is fair, and what happens if every business does the same thing.

Frequently asked questions about algorithmic pricing

Is personalized pricing already happening?

Yes, in limited forms. Lyft has documented systems that personalize driver incentives, and the FTC has found pricing intermediaries with access to detailed behavioural data. However, there is no strong evidence that major companies yet know everyone's precise financial breaking point.

Why is algorithmic pricing a social justice issue?

Because it shifts bargaining power toward firms and away from workers and consumers. Those with less economic agency, including migrants, disabled people, and communities already facing systemic discrimination, are most vulnerable to having their limits extracted and exploited.

What can regulators do?

The FTC's current consultation on personalized pricing is a start. Transparency requirements, such as informing consumers when an offer has been personalised, would help. But deeper change requires challenging the underlying assumption that extracting maximum value from individuals is an acceptable business model.