The machine that learns to watch
Artificial intelligence is usually discussed in two registers, both a little theatrical. In one, it is the herald of abundance: a force that will write code, diagnose disease, tutor children and free humans from drudgery. In the other, it is a phantom of catastrophe: a job-destroying engine, a superintelligence with unclear loyalties, an existential risk wrapped in the glamour of science fiction. Both visions contain truths. But they also obscure the more prosaic, and arguably more dangerous, reality of AI in practice. The technology is not simply learning to think. It is learning to watch.
That distinction matters because the oldest economic appetite in the digital age is not intelligence but data. The core business model of the internet has been to extract fragments of human life, infer what they mean, and sell the ability to influence what happens next. AI supercharges this logic. It does not merely record behavior; it predicts it, shapes it, and increasingly automates the institutions that do the shaping. The result is a new kind of power: less interested in replacing people than in rendering them legible, steerable and profitable. If the first era of the internet was about connection, and the second about attention, the next may be about compliance.
From advertising to behavior management
The phrase “surveillance capitalism” was coined to describe a system that treats human experience as raw material for extraction. Its premise is deceptively simple. The more a platform knows about users, the better it can forecast their actions. The better it can forecast them, the more accurately it can sell access to their future behavior. That model once relied on clicks, likes, searches and browsing histories. AI broadens the aperture dramatically. It can ingest voice, image, movement, location, social graph, biometric signal and text at a scale no human workforce could match. It can then correlate these scraps into patterns that even the people producing them cannot perceive.
This is what makes AI so attractive to powerful institutions. Advertising is the banal surface of the system; prediction is its engine; manipulation is its endgame. The same tools that help a retailer guess what consumers may buy can help a government infer which neighborhoods are politically restive, which workers are likely to unionize, or which online communities deserve closer scrutiny. In private hands, this becomes precision marketing. In public hands, it can become preemptive governance. The difference is not the technology. It is the purpose to which it is put.
There is a temptation, especially among technologists, to insist that these are merely neutral systems reflecting the intentions of users. That argument is too convenient. AI systems are not passive mirrors. They are built within incentive structures that reward accumulation, opacity and scale. Once a firm discovers that more data improves performance, the logic is almost impossible to reverse. Data becomes both the fuel and the moat. The system’s hunger deepens. It reaches into corners of life that once remained private because there was no commercial reason to monitor them. AI supplies that reason.
The automation myth
The other dominant fear is that AI will erase jobs. This is less a prediction than a genre: each industrial revolution is accompanied by anxious prophecies about displaced labor. Some of these worries are real, and some sectors will indeed be battered. Routine white-collar work is especially vulnerable. Drafting, summarizing, scheduling, customer service, basic analysis and administrative coordination are all ripe for partial automation. In many firms, the first response will not be mass layoffs but a subtler erosion: smaller teams, wider spans of control, more work per worker, and a permanently higher expectation of output.
Yet unemployment is not the whole story. The deeper labor effect of AI may be de-skilling and discipline. A software engineer paired with an AI assistant may produce more code, but may also become easier to supervise, easier to benchmark and easier to replace. A journalist using an AI system for research may gain speed, but may also find that editorial judgment is increasingly measured against machine-generated averages. A call center agent with AI support may handle more customers, but under more detailed surveillance of tone, duration and sentiment. Automation is not only a way to eliminate workers; it is a way to reorganize their autonomy.
That reorganization matters because modern labor markets are already defined by insecurity. In such conditions, AI is likely to intensify a familiar asymmetry: firms will keep the gains, while workers absorb the anxiety. Productivity may rise. Wages may not. In theory, machines should liberate people from tedious tasks. In practice, the productivity dividend often accrues to owners and managers before it reaches employees, if it reaches them at all. The political question is not whether AI can do work. It is who gains bargaining power from its deployment.
“The most important effect of AI may not be that it replaces labor, but that it makes labor more observable, more comparable and therefore more governable.”
That governability is the bridge between automation and surveillance capitalism. Once work itself becomes datafied, the workplace can be transformed into a theater of continuous measurement. The gig economy has already previewed this future. Drivers, couriers and delivery workers are not merely assigned tasks; they are scored, nudged and monitored by platforms that know far more about their behavior than any old-fashioned supervisor could. AI extends this regime into office towers, hospitals, classrooms and homes. The ideal worker, from the algorithm’s perspective, is not a person with judgment. It is a pattern that can be optimized.
Why authoritarianism likes AI
Digital authoritarianism does not require jackboot aesthetics or the language of fear. It thrives on convenience, personalization and the administrative efficiencies of prediction. AI is attractive to authoritarian systems for the same reason it is attractive to advertising firms: it lowers the cost of knowing. A state that can cheaply collect, fuse and analyze data across cameras, phones, payments, travel records and social media is a state with a dramatically expanded capacity for social control. It can identify dissidents, map networks, forecast unrest and intervene before organized opposition matures.
But the more insidious version of this power is not overt repression. It is ambient deterrence. Citizens do not need to be punished constantly if they believe they might be watched constantly. They self-censor. They avoid controversial associations. They learn which words are risky. They internalize the machine’s gaze. In that sense, AI does not merely help authoritarianism; it modernizes it. The panopticon has become predictive, and the prisoner is now expected to participate in his own classification.
Democracies are not immune. They may not deploy the full coercive capacity of an authoritarian state, but they are vulnerable to market-led forms of surveillance that erode the same civic habits. When political campaigns microtarget voters with psychologically tailored messaging, when employers monitor productivity at every keystroke, when schools use software to assess student attention, and when police departments buy risk-scoring systems that encode historical bias, the line between governance and manipulation begins to blur. A free society depends on spaces where people can develop opinions, identities and relationships without being constantly translated into data exhaust. AI is making those spaces harder to find.
The problem is not just bias
The public debate about AI often gravitates toward bias, and rightly so. Systems trained on skewed historical data can reproduce discrimination in hiring, lending, policing and healthcare. Facial recognition can misidentify darker-skinned faces more often than lighter ones. Predictive models can misread poor neighborhoods as high-risk simply because they have been overpoliced in the past. These are serious harms. But bias is only one layer of the problem, and arguably not the deepest one. A fairer surveillance system is still a surveillance system.
That is the uncomfortable truth for many reformers. It is easier to ask for better data than to ask whether some forms of data collection should not exist at all. It is easier to demand explainability than to challenge the concentration of informational power itself. Yet if the underlying model rewards ever more intimate extraction, then “responsible AI” can become a fig leaf for expansion. The technology is made safer, perhaps, but also more acceptable, more durable and more deeply embedded.
This is why the policy battle over AI should not be framed solely as a technical question of guardrails. It is a contest over social architecture. What kinds of institutions should be allowed to gather, infer and act on behavioral data? Who gets to decide? What forms of monitoring are compatible with democratic life, and which should be prohibited even if they are efficient? Those are not engineering questions. They are constitutional ones.
The corporate bargain
AI companies present themselves as builders of tools. In reality, many are constructing dependency. Their systems are designed to become indispensable to firms, schools, hospitals and governments that cannot afford to fall behind. This creates a familiar political economy: private companies accumulate public power by embedding themselves in the infrastructure of daily life. They then argue that regulation would stifle innovation, even as their products reshape labor markets, media ecosystems and state capacity.
Such firms also benefit from a strategic ambiguity. They can invoke the promise of general-purpose intelligence when seeking investment and market dominance, then claim to be merely neutral platforms when their tools cause harm. They profit from the language of inevitability. If AI is inevitable, then its social terms appear negotiable only at the margins. But inevitability is usually the most successful marketing slogan in technology. It substitutes destiny for choice.
The choice here is stark. Society can allow AI to evolve as an extension of surveillance capitalism, where each advance in model capability makes human behavior more extractable and more governable. Or it can impose hard limits: data minimization, strict prohibitions on biometric and behavioral tracking, labor protections against algorithmic micromanagement, and real limits on state access to private platforms. That would not halt innovation. It would redirect it.
A more serious definition of progress
There is an old habit in technology policy of treating every problem as a temporary implementation bug. If an AI system discriminates, fix the training set. If it invades privacy, improve consent forms. If it displaces workers, offer retraining. This is reassuring because it preserves the fiction that the system itself is neutral and that only its edges need smoothing. But a technology built to extract value from human behavior will never be morally finished by design tweaks alone.
The question, then, is not whether AI can be used benignly. It can. The question is whether societies are willing to resist the uses to which the most powerful actors will naturally put it. That requires a more mature idea of progress than the one Silicon Valley usually offers. Progress is not merely faster prediction or cheaper labor. It is the protection of autonomy against systems that would reduce human beings to datasets, risk scores and optimization targets.
If the 20th century taught democracies to regulate dangerous industrial technologies, the 21st must teach them to regulate dangerous informational ones. The measure of success will not be the sophistication of the models. It will be whether people still possess zones of life not optimized for sale or surveillance. That may sound modest. It is not. It is the precondition for freedom.
The grander anxieties about artificial intelligence may yet prove justified. Perhaps machines will one day outthink us. But the more urgent threat is that they help institutions manage us before that happens. The future of AI may not be a revolt of the robots. It may be a world in which we comply so smoothly that revolt becomes difficult to imagine.
