The “I” in AI

Submitted to Columbia Bioethics “AI and Ethics” essay contest on 2026-09-30

There is an idea about AI that is held by nearly everyone. It is perpetuated by the very name of this technology, and in nearly every interaction with it. I am talking about “intelligence.” I type into a search engine and a box appears that for a moment, while loading, says “thinking.” I am talking about “thinking.” At every turn, it seems there is a desire to get across to you that humans possess a technology capable of thinking. The proliferation of this idea and its language has become incentivized and ingrained particularly with the rise of Large Language Models (LLMs) in the form of chatbots.
    Let us look at how the idea of thinking technology came into its own. At the start of 1956, this term we now take for granted, artificial intelligence, did not exist. Its genesis is more arbitrary than we realize. John McCarthy coined the term for the title of a conference at Dartmouth College that took place in the summer of that year. According to him, “one of the reasons for inventing the term ‘artificial intelligence’ was to escape association with ‘cybernetics.’ Its association with analog feedback seemed misguided, and I wished to avoid having either to accept… Wiener as a guru or having to argue with him.” (McCarthy 1989). He refers to Norbert Wiener who, in 1948, had published a seminal book coining another new term, cybernetics, derived from the Ancient Greek kybernḗtēs, meaning one who steers a ship (also see, in English: nautical). When using a rudder you must constantly update its positioning in response to the resultant turning of the ship. A cybernetic system is one that is predicated upon feedback loops, where outputs influence new inputs. It is a model with multi-disciplinary scope, from the social sciences to what we now call computer science. McCarthy and the other architects of early AI felt this feedback loop situation, which had great influence among some of those trying to study thinking technology, was too limiting, and they looked to start a field anew, where they wouldn’t feel the burden toward established fields. Historian Ronald Kline writes that “cybernetics split with AI over the issues of ‘symbolic versus continuous systems’ and ‘psychology versus neuro-physiology.’” (Kline 2011). Those astute to the workings of LLMs will likely recognize the reference to “neuro-physiology” as living on in the technology of neural networks, which were in their infancy around the time of the Dartmouth conference. In fact, “the meaning of cybernetics, which initially applied to all approaches to machine intelligence, was increasingly reduced in the US to neural nets and brain modeling after the conference at Dartmouth.” (Kline 2011). Those championing “AI” were part of a movement toward what we might now call symbolic AI, or what philosopher John Haugeland termed, Good Old Fashioned AI (GOFAI). This paradigm sought to model the mind as representations of facts and processes operating on those representations—psychological instead of neural. This divide seems odd to us now because GOFAI is precisely old fashioned. Philosopher of technology Hubert Dreyfus wrote, already in 2007, that it is a “degenerating research program” (Dreyfus 2007). Now, neural networks, much closer to cybernetics than what the original founders of artificial intelligence imagined, have practically become synonymous with “artificial intelligence” in an almost accidental history.
    I would like to distance this discussion from the mere presumption of the impossibility of inorganic, or technological, thinking. The creation of thinking technology is an area that has been studied for decades across many disciplines, and is yet to be conclusive. This discourse has been imperative for computer scientists in the attempt to accomplish artificial intelligence. Dreyfus had critiqued the GOFAI-minded researchers in his 1972 work What Computers Can’t Do, particularly prompted by his MIT colleague, Marvin Minsky, Dartmouth conference co-organizer with McCarthy. Dreyfus’s 2007 article “Why Heideggerian AI failed and how fixing it would require making it more Heideggerian” chronicles this original fallout and the storied proceedings between AI researchers and himself. Dreyfus was a scholar of German philosopher Martin Heidegger. Heidegger put forward a view of being in the world as something more fundamental than how we can make sense of it with mental representations. This is in contrast to René Descartes’ understanding of the mind as separate from the outside world, interacting with it through representations, which was taken in the attempt at symbolic AI (Dreyfus 2007). Dreyfus describes a fundamental problem that thinking technology struggles with, called the frame problem. It describes an AI given a set of facts about the world and imagines a change in state that modifies some of those facts. Determining relevance among changes to the current situation is something that attempts at thinking technology, especially GOFAI, have been unable to manage. In a Heideggerian reading, the relevance of certain facets of the world are not reasoned about mentally, but embodied by the very ways in which we are. Dreyfus follows the field of AI research as it starts to shift toward more neuro-physical rather than symbolic models. He continued to take issue with even these seemingly more Heideggerian models in a way that could be stated as them not being dynamic, or perhaps cybernetic, enough. The act of thinking itself is encompassed in the human way of being in the world, as we are constantly learning and coming to think in unrepresentable ways. On the frame problem, I would argue that while current neural network models present an interesting effectiveness, a glaring gap with human intelligence is found in their biphasic existence of training and inference. Human intelligence is required to construct the informational relevance required for their training. The current technology lacks the way of being that humans have. The only world these neural networks exist in is a temporally-bound and input-bound window of training, after which they are merely prodded with inputs, having their outputs collected (inference). Human intelligence is not biphasic: inference trains. If LLMs are said to possess human-like intelligence, it can only be said insofar as the nervous system’s response to a hitting upon the knee is intelligent. Should we really grant them this term?
    The current technology’s capture by private corporations creates the incentive to act as if this runaway metaphor of intellect were real. The technicians currently most motivated to bring about this technology are steeped in a milieu of science fiction surrounding it. This can be seen in part in the popularity of Roko’s basilisk, a thought experiment originating on an online forum in 2010. It argues that “a sufficiently powerful AI agent would have an incentive to torture anyone who imagined the agent but didn’t work to bring the agent into existence” (Bensinger et al. 2022). While others have a more positive interpretation of what a super-intelligence could bring, it illustrates a sense that exists within the technical community that we are on the precipice of a powerful, permanent alteration to society through the development of artificial intelligence. This message creates huge valuations of what have quickly become some of the most wealthy private companies. OpenAI was founded as a public benefit corporation, and held an ethical board capable of firing its CEO in case it saw its business as getting in the way of its mission to bring benevolent technological intelligence to humanity. In 2024 it did precisely this firing, but within a few days the former CEO, through a number of political pressures, was reinstated, and subsequently lessened the power of the ethical board (Altman and Taylor 2025). The original philanthropic nature that many of these companies had has been subplanted by the incredible amount of wealth they now beget. AI companies’ marketing channels the philanthropic idea of powerful science fiction becoming reality—that they are creating thinking technology. They would like us to believe the changes they are creating in society can be expressed as a mere change in the location of thinking, and enter a world with a preserved, or increased amount of thinking. If these technologies are not actually thinking, they are deluding us into a world with a reduced amount of thinking.
    Considering all of these histories, it is not an extreme leap to state that AI, however it may be employed as a useful technology, currently consists in cybernetic systems that take advantage of the language we ascribe to them in order to self-perpetuate, generate capital, and prevent us from questioning them. My ethical call is for a reframing. The conception, and the confidence in it, that the current technology possesses intelligence and is capable of thinking has political and economic motivations. The opposition to this conception is an ethical urgency. The naming of intelligence gives the current technology a prestige, a level of respect. Arguments around the other areas where this technology may be controversial, be it its energy usage, acquisition of training data, or responsibility, already grant it this benefit in language. This linguistic consideration should be held at the root of questioning anything else about the technology.

Bibliography

Altman, Sam, and Bret Taylor. 2025. “Evolving OpenAI’s Structure.” OpenAI. https://openai.com/index/evolving-our-structure/.

Bensinger, Rob, Miranda Dixon-Luinenburg, Quintin Pope, et al. n.d. “Roko’s Basilisk.” LessWrong. Last edited November 30, 2022. https://www.lesswrong.com/w/rokos-basilisk.

Dreyfus, Hubert L. 2007. “Why Heideggerian AI Failed and How Fixing It Would Require Making It More Heideggerian.” Artificial Intelligence 171, no. 18 (2007): 1137-1160. https://doi.org/10.1016/j.artint.2007.10.012.

Kline, Ronald. 2011. “Cybernetics, Automata Studies, and the Dartmouth Conference on Artificial Intelligence.” IEEE Annals of the History of Computing 33, no. 4 (2011): 5-16. https://doi.org/10.1109/MAHC.2010.44.

McCarthy, John. 1989. “[Review of] Bloomfield, Brian, ed. The Question of Artificial Intelligence.” Annals of the History of Computing 10: 227. https://www-formal.stanford.edu/jmc/reviews/bloomfield.html.