Cell sub-journal critically published that AI is unlikely to generate consciousness in the short term

Original source: NextQuestion

Image source: Generated by Unbounded AI

Interacting with large language models (LLMs), we always have a vague feeling that they may actually be conscious. However, from the perspective of neuroscientists, this view seems to be difficult to hold.

In a recent paper published in Trends in Neurosciences, a sub-journal of Cell, three scholars from computer science, biology and neuroscience delved into the question of "Can artificial intelligence generate consciousness?"

In conclusion, they agree that LLMs cannot be conscious in their current form. How did such a categorical view come about?

▷Source: Cell

LLM & Consciousness

It has long been questioned which animals are conscious, and which entities are conscious besides animals. The recent advent of LLMs has brought a whole new perspective to the problem. It shows us our ability to converse (a manifestation of human consciousness) and makes us redefine and rethink the three concepts of "understanding", "intelligence" and "consciousness".

LLMs are complex, multi-layered artificial neural networks with billions of connection weights trained on tens of billions of words of text data, including natural language conversations between humans. By asking questions through text, the user is drawn into a fascinating simulated context. If you're willing to take the time to use these systems, it's hard not to be blown away by the depth and quality of the network. Ask it a question, and its response is often subtly similar to the one that a conscious individual can produce. Therefore, as an insightful, conscious individual, it is easy to conclude that the answers I receive are generated by an individual who is also "conscious" and capable of thinking, feeling, reasoning, and experiencing. **

Based on the results of such "Turing tests", we can't help but wonder if LLMs are already conscious, or will they soon be conscious? However, this question, in turn, will lead to a series of ethical dilemmas, such as whether it is ethical to continue to develop LLMs that are repeatedly on the verge of "consciousness" awakening? The idea that LLMs are "conscious" is not universally accepted in the neuroscience community today, but as the capabilities of AI systems continue to improve, the discussion of this idea has inevitably come back to the fore. In addition, the major news media are also widely discussing this issue, prompting neuroscientists to interpret the issue objectively from their own professional perspective.

The idea that LLMs are potentially conscious is often supported by an important argument that the architecture of LLMs is largely inspired by the characteristics of the brain (Figure 1), and that the brain is the only object we can confidently attribute to "conscious" at the moment. While early artificial neural networks were designed based on simplified versions of the cerebral cortex, modern LLMs are highly engineered and adapted for specific purposes and no longer retain deep homology to known brain structures. In fact, many of the pathway features that make LLMs computationally powerful (Figure 1) have very different architectures from the systems we currently think have causal power in the generation and shaping of consciousness in mammals. For example, many neuroscience theories related to consciousness generation suggest that the thalamic-cortical system and the arousal system play a central role in the processing of consciousness, which are not available in modern LLMs.

Figure 1: Macroscopic topological differences between mammalian brains and large language models Source: Trends in Neurosciences

At this point, one might ask, is it so important that the architecture of the LLM mimics the characteristics of the brain?

In our opinion, the main reason is that we can only be sure of the existence of one consciousness at the moment, which comes from the brain embedded in the complex body. One might argue that, strictly speaking, this argument might be further narrowed down to just humans, although many of the system-level traits thought to play an important role in subjective consciousness are prevalent throughout the biological spectrum, extending all the way to mammals, even invertebrates.

With that said, let's start with the exact meaning of "consciousness" first. We will then present three arguments against the idea that current AI systems have, or will soon have consciousness in the future, the:

• 1. Consciousness is associated with a stream of sensations that are meaningful to the organism;
• 2. In the mammalian brain, consciousness is supported by a highly interconnected thalamic-cortical system;
• **3. Consciousness may be inseparable from the complex biological organization of biological systems. **

What is consciousness?

Consciousness is a complex concept, and its definition has been debated. In the context of human beings' ability to communicate and interact with each other, the ability to communicate and dialogue is an instinctive element to assess whether a person has consciousness.

Language-based interactive conversations with LLMs often develop an intuitive feeling, which is the starting point for judging whether an LLM is likely to be conscious. However, while LLMs are excellent at interactive conversations, this does not meet the formal objective measure of consciousness, but is only preliminary evidence of intelligence. **

The advent of LLMs has led us to re-evaluate whether a person is able to generate consciousness directly from verbal interactions with others. Therefore, a new view is that we need to reformulate the criteria for judging human-like abilities and human-like characteristics.

The word "consciousness" often has different meanings. For example, neurologists often refer to the "level of consciousness", which is the first assessment of whether a person is conscious and then the level or specific state of consciousness in a more granular way. Psychologists, on the other hand, are more concerned with the content of consciousness: the specific experiences, memories, and thoughts of an individual's inner world. In addition, there are differences between the different contents of consciousness. Our experience can be described as phenomenal or experiential (e.g., seeing or smelling an apple, or touching your arm) or in more abstract form (e.g., how we imagine, envision, or manipulate conceptual memory).

The question of whether an AI system is conscious can be answered in a number of ways: it can focus on some of the meanings of consciousness, or it can focus on all the meanings of consciousness at the same time. In the following, we focus primarily on phenomenal awareness and explore whether machines are capable of experiencing the world phenomenally.

About the Environment

The part of an organism that can be used in the process of perceiving the outside world is called its environment. For example, the human retina responds to light with wavelengths of 380 nm – 740 nm, i.e., the retina is able to perceive the spectrum from blue to red. Without the aid of external technology, humans cannot detect infrared light (>740 nm) or ultraviolet light (< 380 nm) outside of this wavelength range. We also have similar environments in terms of hearing, somatosensory, and vestibular sense, namely the corresponding auditory domains (the human ear can hear sounds from 20 Hz – 20,000 Hz), the somatosensory domains (humans can distinguish stimuli within about 1 mm of certain parts of the body), and the vestibular domain (the interconnected 3D structures of the human semicircular canals provide us with an inner sense of balance). At the same time, other species in nature are able to detect signals in other bands of the electromagnetic spectrum. For example, bees can see light in the ultraviolet range, and snakes can detect infrared radiation signals in addition to more traditional visual signals.

That is, different animals have different sensitivities with which their bodies and brains are able to perceive their surroundings. Gibson, an American psychologist, refers to the possibility of an organism acting in a particular environment as "affordance" (with the penetration of Internet technology, affordance began to be used to explain the use of digital technology in media practices and human daily interactions). **

According to the nature of its algorithm design, LLMs only have binary encoding patterns, can only receive binary information input, and further execute network algorithms inherent in complex transformer structures, which constitute the working architecture of today's LLMs. While neuronal spikes are also capable of encoding incoming analog signals into digital signals (i.e., binary signals), the flow of information delivered to the LLM is highly abstract and does not have any strong connection to the outside world itself. Text and speech encoded as a string of letters simply cannot match the dynamic complexity of the natural world, i.e., the environment of an LLM (the binary information provided to it) is fundamentally different from the information that enters our brains when we open our eyes or communicate conversations, and the experiences that come with it. Traditional philosophical discourse emphasizes the uniqueness of the flow of information between different species (e.g., the difference between humans and bats) and the phenomenological characteristics of these experiences. We believe that the information input obtained by LLMs may exhibit more significant differences, although there is no definitive way to quantify this difference for the time being.

That being said, the input of AI systems will become inexorably richer in the future. Future LLMs can be equipped with different types of inputs that can better match the types of signals that conscious agents can access on a daily basis (i.e., statistics from the natural world). So, will the available environment of AI systems in the future be wider than the human environment?

In answering this question, we must recognize that the human subconscious and conscious experience is not solely determined by sensory input. For example, imagine that when we lie in a pontoon, we are still conscious despite our lack of normal sensory experience. The concept here is highlighted that the environment presupposes an inherent subjective perspective, that is, to start from a subject. Similarly, afforfority depends on the internal nature of the subject, in particular the subject's motivations and goals. This means that consciousness cannot be generated by the environment alone (the input data of the LLM). Therefore, simply feeding a large stream of data into an AI system does not make the AI system itself conscious. **

This perspective may prompt us to rethink some of the basic assumptions in the science of consciousness. Specifically, as AI systems gradually exhibit increasingly sophisticated capabilities, researchers will have to re-evaluate the necessity of the more fundamental self and agent-related processes proposed by certain theories of consciousness for the emergence of consciousness.

**"Integration" of Consciousness **

At present, there have been many studies on the neural correlation of consciousness, among which there are many different theories about the neural circuits of consciousness processing. Some emphasise that consciousness is underpinned by a dense, highly connected thalamic-cortical network. **The thalamic-cortical network includes cortical regions, cortical-cortical junctions, and divergent projections of higher thalamic nuclei to cortical regions. This specific structure of the thalamic-cortical system supports circulatory and complex thought processing that underpins consciousness and conscious integration (i.e., consciousness is unified despite the fact that consciousness arises from different brain regions). However, different theories hold different views on the way to achieve the integration of consciousness.

According to the global neuronal workspace theory (GNW), consciousness relies on a central workspace consisting of a distributed frontoparietal cortex system. This workspace integrates information from local cortical processors and then transmits it to all cortical local processors on a global scale, with global delivery separating conscious and unconscious processes. Other theories of consciousness hold that conscious integration is achieved by other neural processes. For example, neuronal dendritic integration theory (DIT) suggests that conscious integration occurs through a high-frequency synchronization phenomenon between different cortical regions, which may involve different functions including perception, cognition, or motor planning, depending on the cortical region involved.

Figure 2: Neural structure of consciousness integration based on the theory of neuron dendritic integration (DIT) Source: Trends in Neurosciences

*Caption: In the DIT theory (Figure 2), the researchers believe that global conscious integration also depends on local integration of pyramidal neurons in the fifth layer of the cortex, a large excitatory neuron that is central in both the thalamic-cortical and cortical circuits. There are two main structures in this type of neuron (Figure 2, orange and red cylinders) that process completely different types of information: the basal structure (red) processes the external basic information, while the apical structure (orange) processes the information generated internally. According to DIT theory, in the state of consciousness, these two structures are coupled to each other, allowing information to flow through the thalamic-cortical and cortico-cortical circuits, thus enabling system-wide integration of information and consciousness generation. *

It is important to note that the architectures of today's LLMs and other AI systems lack the features that these theories emphasize: existing LLMs have neither equivalent bistructural pyramidal neurons, nor centralized thalamic architectures, global workspaces, or multiple features of ascending-arousal systems. In other words, existing AI systems lack the brain features that are currently believed by the neuroscience community to underpin consciousness generation. Although the mammalian brain is not the only structure capable of supporting the production of consciousness, evidence from neurobiology suggests that the formation of mammalian consciousness is determined by very specific structural principles (i.e., simple connections between integrated and excited neurons). Topologically, the structure of existing AI systems is extremely simple, which is one of the reasons why we do not consider existing AI systems to be phenomenal-aware.

So, will future AI models finally be able to integrate the process of "integration" that many theories of consciousness regard as the core? In response to this problem, the concept of "integration" proposed by the GNW theory provides a relatively simple way to implement it. In fact, some recent AI systems have been incorporated into something like a global workspace shared by a local processor. Since the computational process of global transmission can be implemented in an AI system, according to this theory, an AI system that adopts this computational method will contain the core components of latent consciousness.

However, as mentioned earlier, not all theories of consciousness agree that this mode of integration is the key to the generation of consciousness. For example, the integrated information theory of consciousness argues that software-based AI systems implemented on a typical modern computer cannot be conscious because modern computers do not have the proper architecture to achieve the causal reasoning capabilities needed to fully integrate information. Therefore, we will consider the third possibility, which is that consciousness is achievable in principle, but it may need to go beyond the current (and perhaps future) level of computational specificity of AI systems. **

Consciousness is a complex biological process

The generation of consciousness does not depend only on the architecture of the system. For example, when we are in deep sleep or anesthesia, the structure of the thalamic-cortical system does not change, but consciousness disappears. Even in deep sleep, local neural responses and gamma belt activity in major sensory areas are similar to those in the conscious state. This suggests that consciousness relies on specific neural processes, but these neural processes are different in conscious and unconscious brains. **

To shed light on the detailed differences between conscious and unconscious processing, let's first go back to the theory of neuron dendritic integration (DIT). The DIT theory contains a number of neurobiological nuances related to neural processes that are consciously and unconsciously processed. The DIT theory proposes that the key difference between conscious and unconscious processing lies in the integration of the two compartment structures of pyramidal cells (Figure 2). As mentioned earlier, during conscious processing, these two structures interact with each other, allowing the entire thalamic-cortical system to process and integrate complex information. However, in the anesthetic state, various anesthetics lead to functional decoupling between the two structures of vertebral neurons. In other words, although these vertebral neurons are anatomically intact and can excite action potentials, their dendritic integration capacity is severely limited physiologically, i.e., top-down feedback cannot influence processing. Studies have shown that this dendritic coupling is controlled by metabotropic receptors, however this structure is often overlooked in computational models and artificial neural networks. In addition, studies have shown that in this case, the higher thalamic nuclei control the activity of this metabotropic receptor. Thus, specific neurobiological processes may be responsible for "turning on" and "off" consciousness in the brain. This suggests that the quality of experience in the mammalian brain has an intricate relationship with the underlying processes that produce consciousness. **

While these theories are convincing enough, it is almost certain that this knowledge pales in comparison to the complexity of the neural processes that arise from a complete understanding of consciousness. Our current explanations of consciousness rely on theories such as global workspaces, integrated information, circular processing, dendritic integration, etc., but the biological processes by which true consciousness arises may be much more complex than currently understood by these theories. It is even quite possible that the abstract computational-level ideas currently used to construct the discussion of consciousness research may have completely failed to take into account the necessary computational details required to explain consciousness.

In other words, biology is complex, and our current understanding of biocomputing is limited (Figure 3), so perhaps we lack the right mathematical and experimental tools to understand consciousness. **

Figure 2: Neural structure of consciousness integration based on the theory of neuron dendritic integration (DIT) Source: Trends in Neurosciences

*Caption: In the DIT theory (Figure 2), the researchers believe that global conscious integration also depends on local integration of pyramidal neurons in the fifth layer of the cortex, a large excitatory neuron that is central in both the thalamic-cortical and cortical circuits. There are two main structures in this type of neuron (Figure 2, orange and red cylinders) that process completely different types of information: the basal structure (red) processes the external basic information, while the apical structure (orange) processes the information generated internally. According to DIT theory, in the state of consciousness, these two structures are coupled to each other, allowing information to flow through the thalamic-cortical and cortico-cortical circuits, thus enabling system-wide integration of information and consciousness generation. *

In order to better understand biological complexity, it is important to emphasize that the biological processes described above at the cellular and systemic levels must occur in a living organism and are inseparable. Living organisms differ from today's machines and AI algorithms in that they are able to constantly maintain themselves at different levels of processing. In addition, living systems have a multifaceted history of evolution and development, and their existence depends on their activities at multiple organizational levels. Consciousness is intricately linked to the organization of living systems. It is worth noting, however, that today's computers are not capable of embodying this organizational complexity of living systems (i.e., the interaction between different levels of the system). This suggests that modern AI algorithms do not have any organizational level constraints and cannot work as effectively as a living system. This means that as long as AI is software-based, it may not be suitable for being conscious and intelligent. **

The concept of biological complexity can also be expressed at the cellular level. A biological neuron is not just an abstract entity that can be fully captured with a few lines of code. In contrast, biological neurons have a multi-layered organization and rely on further cascades of complex biophysical processes within neurons. Take the "Krebs cycle", for example, which is the basis of cellular respiration and is a key process in maintaining cellular homeostasis. Cellular respiration is a critical biological process that enables cells to convert energy stored in organic molecules into a form of energy that cells can utilize. However, this process cannot be "compressed" into software, as biophysical processes like cellular respiration need to be based on real physical molecules. Of course, this does not mean that consciousness needs a "Krebs cycle", but rather emphasizes that similar challenges may be involved in the process of understanding consciousness, i.e., perhaps consciousness cannot be detached from the underlying mechanism. **

However, we do not fully agree with the claim that consciousness cannot be generated by intelligent systems at all, but we must consider the correlation between consciousness and the complex biological organization behind life, and the types of computations that capture the nature of consciousness may be much more complex than our current theories understand (Figure 3). It is almost impossible to perform a "biopsy" of consciousness and remove it from the tissue. This view contradicts many current theories about consciousness, which hold that consciousness can arise at an abstract computational level. Now, this assumption needs to be updated in light of modern AI systems: in order to fully understand consciousness, we cannot ignore the cross-scale interdependence and organizational complexity observed in living systems. **

Although AI systems mimic their biological counterparts at the level of network computing, in these systems all other levels of biological processes have been abstracted away from the processes that have a close causal relationship with consciousness in the brain, so existing AI systems may have abstracted consciousness itself. As a result, LLMs and future AI systems may be trapped in an endless stream of simulated consciousness features, but without any phenomenal consciousness to speak of. If consciousness is indeed related to these other levels of processing, or to their interaction between different scales, then we are far from the possibility of a machine generating consciousness.

Summary

Here, we explore the possibility of consciousness in LLMs and future AI systems from a neuroscientific perspective. As attractive as LLMs are, they are not conscious and will not be conscious for a shorter period of time in the future.

First, we illustrate the vast difference between the environment of mammals (a "small fraction" of the external world they can perceive) and the highly impoverished and limited environment of LLMs. Second, we argue that the topology of LLMs, while very complex, is empirically very different from the neurobiological details of mammalian consciousness-related circuits, and therefore there is no good reason to think that LLMs are capable of generating phenomenal consciousness (Figure 1). It is not yet possible to abstract consciousness from the complexity of biological organization, which is inherent in living systems, but which obviously does not exist in AI systems. Overall, the above three key points make it impossible for LLMs to be conscious in their current form. They mimic only the characteristics of human natural language communication used to describe the richness of conscious experience.

Through this article, we hope that the arguments presented will have some positive impact and reflection (see Unresolved Questions) and not just represent an objection. First, the current potential ethical concerns about the perceived capacity of LLMs are more hypothetical than real. In addition, we believe that a deeper understanding of the similarities and differences between LLMs and mammalian brain topologies can advance advances in machine learning and neuroscience. We also hope to advance the machine learning and neuroscience community by mimicking the characteristics of brain tissue and learning how simple distributed systems process complex information flows. For these reasons, we are optimistic that future collaborations between AI researchers and neuroscientists can lead to a deeper understanding of consciousness.

Unresolved Follow-up:

• 1. Awareness assessment in LLMs and AI often relies on language-based tests to detect consciousness. Is it possible to assess consciousness based solely on language (i.e., text), and are there further evaluative features that can help determine whether an artificial system is conscious?
• 2. The neural basis of mammalian consciousness is related to the thalamic-cortical system. How can the thalamic-cortical system be implemented in AI? What specific functions and tasks would benefit from a thalamic-cortical system?
• 3. The ascending-arousal system also plays a crucial role in the generation of consciousness in organisms, and it plays a complex and multifaceted role in shaping neurodynamics. To what extent does AI need to mimic these different processes in order to reap the computational advantage of the rising awakening system?
• 4. In addition to the thalamic-cortical system, dendrites play a key role in some of the theories of consciousness discussed in this article. Is dendrites just one factor that increases the computational complexity/efficiency of biological neural networks, or is there more to it?
• 5. Is the organizational complexity of living systems related to consciousness? Living systems are made up of different levels of processing processes that interact with each other. Can the organizational complexity of living systems be explained in more complete detail? Are new mathematical frameworks needed to deal with such systems in order to shed more light on the biological processes by which consciousness arises?
• 6. Some theories suggest that consciousness and agency are inextricably linked. To understand how consciousness arises from biological activity, does it need to first understand agency?

Original link

• Aru, J., Larkum, M.E. and Shine, J.M. (2023b) ‘The feasibility of artificial consciousness through the lens of neuroscience’, Trends in Neurosciences [Preprint] . doi:10.1016/j.tins.2023.09.009.