The machine that learns us back

Artificial intelligence is usually described as a tool for making machines more intelligent. That framing is convenient for the companies selling it, but incomplete in the way a slogan is incomplete. The more consequential fact is that AI is increasingly a tool for making institutions more powerful. It helps employers monitor workers, platforms anticipate attention, governments classify populations and advertisers profit from human prediction. In that sense, AI is not merely automating labor. It is automating power.

This is why the standard debate about AI job displacement is too narrow. The obvious question is how many jobs will disappear, and which occupations will be transformed first. The larger question is who gets to see, measure and shape human behavior at scale. Once AI becomes good enough to infer moods, habits, performance and susceptibility, it stops being just a technology of efficiency. It becomes a technology of governance. Researchers and policy experts have warned that AI can supercharge the extraction and use of personal data, blurring the line between public and private life while making decisions about people’s lives less visible to them.[1][2]

The deepest danger, then, is not that machines will wake up and replace us. It is that institutions will use machine intelligence to know us more intimately than we know ourselves, and to convert that asymmetry into profit and control.

From automation to behavioral extraction

For most of the industrial era, automation meant replacing muscle. The assembly line displaced artisans, and software replaced clerical routines. AI is different because it increasingly targets judgment, prediction and management. It can rank résumés, score customers, optimize delivery routes, monitor call-center agents and set the pace of warehouse labor. In every case, the promise is the same: fewer humans, more output.

But that promise obscures a second-order effect. AI systems thrive on data, and data are not neutral raw material. They are the residue of human life, collected through search queries, location trails, workplace keystrokes, social interactions and consumer habits. Once collected, that data rarely stay confined to the narrow purpose for which they were supposedly gathered. As one policy expert put it, once organizations begin collecting data, they are almost never going to stop.[1] The result is a self-reinforcing machine: the more AI systems observe, the more they can predict; the more they predict, the more valuable observation becomes.

That is the logic of surveillance capitalism. It is not simply advertising with a fancier interface. It is an economic model built on the secret extraction of behavioral data and the translation of that data into commercial advantage and social influence.[2][5][7] Under that model, AI is not an accidental byproduct. It is the accelerant.

As Maria Ressa and other critics have argued, AI is effectively a surveillance derivative: it depends on the data exhaust of digital life, the infrastructure built to store and process that exhaust, and the feedback loop that makes surveillance more profitable the more comprehensively it is deployed.[3] In other words, AI does not merely live inside the surveillance economy. It deepens it.

The labor market’s false comfort

Executives often speak of AI-led productivity gains as if they were a public good that happens to enrich shareholders. Economists, meanwhile, tend to debate whether automation destroys more jobs than it creates over the long run. Both perspectives can be true and still miss the political economy of the transition.

The first problem is timing. Job displacement can arrive faster than new work can emerge. Entire occupational categories do not simply vanish; they are thinned, deskilled and made more precarious. AI systems can enable firms to hire fewer people, demand more from the people they keep and fragment work into managed tasks that are easier to monitor and easier to outsource. That shift has consequences beyond employment statistics. It weakens bargaining power, compresses wages and changes the balance between labor and capital.

The second problem is selectivity. AI rarely eliminates all jobs equally. It tends to hollow out middle-skill roles, intensify low-wage service work and reward a smaller class of highly paid designers, engineers and owners. That makes the technology not just disruptive but distributive: it moves income upward unless countered by policy, collective bargaining or regulation.

The third problem is that productivity gains do not automatically become social gains. A firm may use AI to write more code, answer more customer queries or draft more legal memos. It may not use the resulting surplus to shorten the workweek, raise wages or invest in worker transition. In practice, many organizations will do the opposite: they will treat AI as a justification for doing the same work with fewer people and more surveillance of those who remain.

Automation does not only remove tasks. It also changes who has power over the tasks that remain.

Workplaces as laboratories of compliance

Nowhere is this more visible than in the AI-managed workplace. Warehouses track pick rates and pauses. Delivery platforms assign routes and measure motion. Office software watches typing speed, app use and calendar behavior. Customer-service systems analyze tone and response time. The stated purpose is efficiency. The practical effect is often discipline.

This is not a marginal issue. When AI is used to evaluate workers continuously, it transforms management from human judgment into statistical control. Workers are not only told what to do; they are told what the algorithm expects them to do next. In that environment, the boundary between productivity and surveillance disappears. The employee becomes both labor and data source.

Harvard policy discussions have already pointed to the need for stronger privacy protections for workers subject to automated management, along with broader democratic oversight of high-risk systems.[2] That emphasis matters because workplace AI is an early warning for the rest of society. The same systems that track a warehouse shift can be adapted to monitor classroom attention, police patrols or welfare recipients. Once automated monitoring becomes normal in one sphere, it becomes easier to justify elsewhere.

Surveillance capitalism’s political logic

The political danger of AI is not limited to the private sector. Its methods travel well. Governments that want to regulate populations more tightly can borrow the same logic: collect more data, infer more intent and intervene earlier. In the most advanced cases, the result is digital authoritarianism: a state that does not merely watch dissidents but predicts, sorts and neutralizes them before dissent becomes visible.

That is why critics of surveillance capitalism see it not as a business model alone, but as a democratic hazard. If platforms learn to predict human behavior for profit, political actors will inevitably try to use similar methods for persuasion and control. The boundary between targeted advertising and targeted propaganda is thinner than the industry likes to admit. AI makes that boundary thinner still.

One reason is scale. Human censorship is expensive and visible; algorithmic censorship can be cheap, dynamic and hard to audit. Another reason is precision. A system that knows which users fear inflation, which ones distrust migrants and which ones are vulnerable to outrage can tailor messages accordingly. Political manipulation becomes more efficient when it is personalized. That does not mean democracy collapses overnight. It means the informational conditions for democratic choice erode gradually, while citizens continue to believe they are choosing freely.

In this context, the most seductive claim about AI—that it can predict what people want—becomes the most dangerous. Prediction is not neutral when it is used to shape preferences, limit options or suppress dissent. The issue is not that AI knows the future. It is that institutions can use its predictions to manufacture the future they prefer.

Why regulation keeps arriving late

Policy responses have lagged because AI is routinely framed as an inevitability. Governments are told that strict rules will slow innovation, drive firms overseas or leave democracies behind in a race they cannot afford to lose. This is a familiar argument from every previous era of unregulated technology. It is also a partial truth weaponized into a political strategy.

Regulation is harder than rhetoric because the harms are diffuse. A factory accident is visible; a manipulated attention span is not. A layoff is counted; a worker’s lost autonomy is not. A discriminatory algorithm can be buried inside proprietary code, shielded by trade-secret claims and defended as a statistical model. That opacity gives AI companies a structural advantage over regulators and the public.

Some analysts have proposed more aggressive responses: a national registry for high-risk AI systems, stronger privacy protections, and broader tests of whether new technologies expand or undermine civic capacity.[2] Others argue that the core logic of surveillance capitalism must be dismantled, not merely managed, because the business model itself depends on extraction at scale.[2][7] Whether one prefers reform or abolition, the underlying diagnosis is the same: if data extraction remains the engine, AI will keep incentivizing surveillance.

That is why consent banners and privacy settings are not enough. A user clicking “accept” does not meaningfully consent to a system that infers personality, predicts behavior and monetizes vulnerability. Nor does a worker agreeing to company software fully understand the extent to which performance metrics can harden into automated discipline. Effective regulation must therefore move upstream, governing collection, retention, use and secondary use of data, rather than merely punishing abuse after the fact.

The politics of resignation

The most troubling feature of the AI era may be cultural rather than technical: resignation. Many people already assume they are being tracked, nudged and scored. They have stopped expecting privacy and started adapting to surveillance as the price of modern life. This matters because surveillance capitalism depends not only on data collection, but on normalized extraction. A population that believes it cannot opt out is easier to manage.

Yet resignation is not inevitability. Digital systems are built by institutions and maintained by policy choices. They can be designed differently. AI can be used for genuinely public purposes: detecting disease, improving accessibility, translating languages, optimizing energy systems and reducing dangerous drudgery. But those uses will not emerge by accident if the dominant commercial model continues to reward extraction over accountability.

The more sobering possibility is that AI will not produce a single dramatic rupture. It will produce a thousand smaller accommodations: workers monitored more closely, consumers profiled more deeply, citizens targeted more precisely, dissent filtered more efficiently. Each change will look manageable in isolation. Together they will reshape the terms of freedom.

That is what makes AI such a potent political technology. It does not need to command people directly. It only needs to understand them well enough to steer them. In the old industrial economy, power was visible in factories and offices. In the new one, power is increasingly hidden in models, metrics and databases—quietly deciding who gets hired, who gets watched, who gets persuaded and who gets ignored.

The central question is therefore not whether AI will replace some jobs. It will. The real question is what kind of society emerges when prediction becomes a business model and surveillance becomes infrastructure. If that question is left to the firms best positioned to profit from the answer, the future will not be automated. It will be administered.