Thoughts on Evaluating Consciousness

  • Sentience: the capacity for phenomenal experience or qualia, such as the capacity to feel pain and suffer
  • Sapience: a set of capacities associated with higher intelligence, such as self‐ awareness and being a reason‐responsive agent.

(From “The Ethics Of Artificial Intelligence” Nick Bostrom, Eliezer Yudkowsky)

Jump to evalutation test ->

Intro

The idea of machines reaching Artificial General Intelligence (AGI) or Human-Level Intelligence has become a popular subject since the inception of computers. In recent years, with the accelerated progress in machine learning approaches and the success of deep neural networks and their variants, we see new science methods, daily applications, disrupted industries based on these approaches at their core. We are evident to huge progress, part of it is indirectly and part directly dedicated to achieving Human-Level Intelligence machines. Can more compute / data / modified and larger models could do the job?

As part of the AGI discussion, the question of a conscious machine is brought up quite often. While many have tried to define consciousness properly and even calculate it, the main point that I’m having trouble wrapping my head around is whether consciousness is independent of intelligence or not. Is it a trait of an intelligence system or independent phenomena of a physical system. If we define consciousness as “what we experience?” we might be left with too broad of a definition, but if we separate it to self-awareness 1 and qualia 2, we might be left with a conflict. Can self-awareness exist without feeling it? Is it possible to have a physical system with self-awareness only, and if so, why isn’t it a good enough criterion for a conscious machine?

The question of a conscious machine is far from being a new one. Discussions on the topic can take us back to John Searle and his “Can Computer Think?” chapter to new implications of the Integrated Information Theory (Giulio Tononi (2015)) claiming digital computers are incapable of actual experience. John Searle conclusion from The Chinese Room argument is that a digital computer is incapable to “really” think. His thought experiment implies that computers merely use syntactic rules to manipulate symbol string, but have no understanding of meaning or semantics. Christof Koch uses the Integrated Information Theory (IIT) to argue that Intelligence and consciousness are independent of each other and comes to the conclusion that current digital machines cannot be conscious. According to IIT, consciousness is determined by the causal properties of any physical system acting upon itself, IIT computes the associated integrated information to determine whether that system feels like something. On the other hand, some scientists think that consciousness is independent of intelligence, Jeff Hawkins, the author of “On Intelligence”, argues exactly that. To him, consciousness is part of intelligence, “it is how it feels to have a neocortex”, inseparable from intelligence itself.

To be honest, I’m torn from both sides. It is hard for me to believe consciousness can be separated from intelligence 100%, leading us to the ancient school of thought: panpsychism. Can it be that wrong definitions or axioms support wrong proofs on what is consciousness and what has consciousness? To my understanding IIT doesn’t make the differentiation between sapience to sentient, but would it be possible to be self-aware without qualia ? a machine with a self-awareness that just lacks the feeling capacity that we have ? or is it a contradiction in itself. Maybe these definitions are just too mixed up ..

These answers are yet to be solved and there is still a cloud of uncertainty that surrounds it. Maybe we can approach the problem as engineers ?

Will an AGI have consciousness and how could we evaluate it ?


What is an AGI?

AGI by definition is a super-smart agent that can generalize on the world data and imitate almost every behavior of human beings, to the point where we could hardly differentiate between a human or such agent in a simple texting conversation, aka. The Turing Test (other tests are available). Whether we will reach AGI and how we’ll reach it is a different discussion. Deep nets and modifications of it have taken us far and will probably take us furthermore. But even if we’ll achieve AGI with these kinds of models, we are still far behind when it comes to explaining these models’ behavior.


Is it important ?

As I mentioned, Jeff Hawkins defines consciousness in his book “On Intelligence” as “consciousness is simply what it feels like when you have a cortex”. Why then, is it so important to know if a machine’s brain will have such quality or not? Why do we need to understand it? If we can just imitate the neocortex, doesn’t it mean we can create consciousness? if so, how should we go about treating and interacting with it? There is a big ethical question we can dive into, but I will avoid going in this direction and will prefer another avenue of thought, man-computer symbiosis.

Ray Kurzweil is talking about uploading our consciousness to a supercomputer. The day when nanorobots will crawl within your brain, recording every synapse and every connection, and then report all the information to a supercomputer, which will reconfigure itself into you! If we assume it can be done, is it possible that our minds computed on external machines could feel? if not, and it is just a high capacity function of our cortex, what happens when the mind is uploaded back to us? aren’t they turned to experiences? how can we test it? evaluate it? These kinds of questions led me to think about consciousness and evaluation.


Consciousness

how do we evaluate consciousness ? and do we really need to formulate what is consciousness in order to evaluate it ?

If we think about it, the answer to the second question is quite easy. We are a living example of beings that can evaluate our consciousness state without really understanding how to formulate what it is. I can ask you to meditate for a few minutes and make sure you are aware of yourself, aware of the sensations and your surroundings, you’ll find out that you can classify this experience to a conscious experience quite easily. Even if you don’t know how to explain it or formulate it mathematically. Moreover, I could ask you to classify a non-conscious experience you had, and you will probably manage to pull this off as well. Let me try to give you a simple example of a non-conscious state as it is a key part of the essence of this essay.

This might be hard for people that never been intoxicated from alcohol before or tried other drugs such as cannabis, but I’ll try to do my best. Anesthesia - the feeling when taking drugs, when you feel the level consciousness is moving down. When you try cannabis for example, and you passed your “nice to have” limit, you get into a state where your short-term memory is vague, you might get dizzy a bit and overall you might feel like you are not 100% yourself. When you are in this state, you get these flashes or blips of realization where you like “oops, I was really not conscious for a few seconds”. It is like you are getting dumber and dumber and you have these blips of High “IQ” states or self-awareness where you understand how dumb are you and how not aware you were to yourself in that experience. In a way, my point is that part of being self-aware sentient is having the ability to classify a non-conscious experience as well. Something that we can do when we self reflects on these kinds of experiences. I do want to emphasize that the level of awareness is the one damaged more and not the qualia (at least from my view), so if we have the level of self-awareness, can’t other physical systems have it, independent of the actual sensation feelings?

How can we evaluate consciousness then? In practice, there are actually several methods to evaluate whether a human being is in a conscious state or not, like the Zap and Zip technique. A technique taking advantage of the fact that a conscious brain will display much more complex, unpredictable electrical activity than one that is unconscious, which response with simple, regular patterns. The problem we face is that AGI isn’t necessarily going to be hosted on a biological body. We can treat our brain as a black box and try to identify patterns through experience, but it doesn’t give us a true answer on consciousness, it gives us the ability to evaluate ourselves (humans). The Integrated Information Theory (IIT) evaluates consciousness of any physical system, a number between 0 to 1. But the IIT is a theory for computing consciousness based on its own defined axioms and postulates. Maybe we can try another approach? A more machine-learning-like approach for evaluation ..


My suggestion

Disclaimer

  • My approach is naive, I would treat it as a guideline to steps toward research paths.
  • We can always go back to a more simplified version of this test.

The Test:

If I am a conscious being, given a non-conscious and a conscious experience, I could probably classify these experiences with good accuracy. Using this as a premise, here is my science fiction test for evaluating if an AI agent is conscious or not.

  1. Prepare a dataset of real human beings’ experiences.
    • Each experience can be classified accurately by the human that had this experience.
  2. Deploy the dataset to the agent that you want to test.
  3. For each experience, run it on the agent and ask the agent if it was conscious or not.
  4. (Optional) ask the agent to generate conscious experiences and run them on yourself.

Obviously, it is the most naive approach I could think of, but it can be a useful thought experiment to lead interesting research avenues. Let me try to dive into each step and convince you.


Step A - Prepare a dataset of real human beings experiences.

Imagine we could encode our experiences and transfer these experiences between us, or deploy these experiences in other computational forms. It might seem crazy, but we’re already doing it in a sense. When we describe experiences we put them in words and it gives us the ability to message our experience to the world. Others can read it and understand it or sympathize with the story on different levels. The main problem with this kind of encoded experience is that we lose a lot of information on the way. Making progress to find a better encoding with less reduction of data can get us closer to the first step of the utopian test I suggested.

To form a dataset we first need the ability to record an experience - our first obstacle to overcome. If everything in the brain just patterns, we might create a brain recorder to store and decode them later to recall our experiences. Although it will be very hard to test patterns from an experience and to save these patterns, it can still make a good alternative. But we still need to understand how our brain stores these patterns exactly, the Thousand Brains Theory of Intelligence is a good start.

Another approach will be to record the experience as it is happening - our brain activity in “live”. A naive approach will try to record a “5 seconds” experience of all the neural activity in our brain. The problem with this approach is that even if we had a method to recreate the record in our brain, other’s brains are different, meaning there is no one-to-one neuron in your brain to every other human brain. The connections that neurons form during our lifetime are different between us - making our brain not so suitable for a DVD kind of applications. This means we need a half-way approach to abstract our live experiences or any physical system experiences away from the actual machinery to be able to communicate them.


Step B - Deploy the dataset to the agent that you want to test.

Let us say that we manage to store an experience in a meaningful way, and even manage to transfer these kinds of experiences between human-like brains. This does not guarantee us a meaningful transfer of experiences between us to computer-like brains. How can we imitate an electric activity of our brain to silicon compute brains with entire different architecture behind it?

The problem with actual computers, especially non-biological ones, is that their architecture is entirely different than ours. Recording experiences as they are is not enough, we need to find some sort of a medium to transfer these experiences between us to non-human-like brains. In a sense, we must find a way to abstract our experiences without losing meaningful patterns of data (like words), a way to abstract our experiences in a level that will be enough to deploy this experience to an actual computer. Similar to how you can imagine someone mimicking the neural behavior of someone’s else brain into your brain, but just instead of you, do it to a computer.

I believe this will be the most difficult part of the test (if possible). Not only it will require us to understand our human brain activities in such depth that we could understand its own abstractions. But we will need to understand it in our machines as well, something we are having difficulties with already today. Explaining the actions of modern AI architectures is not obvious at all.


Step C - For each experience, run it on the agent and ask the agent if this experience was with consciousness or not.

Abstracting our consciousness doesn’t mean we know how to represent it in other forms of computation. It is probable that we could record patterns from our brain and manage to abstract them in a way that will be useful to “read” for other human-like brains. This will not necessarily require a deep understanding of what is conscious and all the avenues of what is the object we abstracted. On the other hand, if we want to make these abstraction transferable to machines (digital ones), I don’t see other ways than:

  • Find the function to transfer these patterns to a readable machine form.
  • Build the AI with a similar hierarchical structure of human brains.

The third part of the test is focusing on the actual subject, how can we execute the experience in the subject of the test. This is highly related to the other parts of the test, but it is not obvious as well. Even if we could store experiences and deploy them in a meaningful way, execute them on a subject can be viewed as the hardest part of the test. How do we copy a human experience in ours? We talked about mimicking the neural activity, but are there any other ways?


Aftermath notes

  • Can agents cheat the test?

    First of all, of course, but it depends. If we assume all of our technical problems from the suggested test are solvable, we can assume that such a dataset of experiences is reachable. An agent can always pre-train on this dataset and might learn to classify correctly the different kinds of experiences. Nevertheless, the idea is to create an evaluation test only. Assuming we can reach AGI without actually recording human experiences, we don’t want to let these agents to study our datasets. A dataset will only be accessible at evaluation time.

  • The test raises a lot of issues that I yet to have answers, the biggest for me is the following:

    If we managed to build such an evaluation process, does it implies that we can actually build a conscious computer? if so, is the test even relevant?

  • If we can simulate a conscious experience in a computer-like brain, then how can we say that this computer doesn’t understand consciousness? we don’t. Our test is quite simple in that regard, the machine should classify non-conscious states as ones, and should generate other experiences to be deployed for humans to evaluate, the reverse approach.
  • My test is the naive approach on how we can evaluate if a machine is sapient or not. Sentience detector might be a side effect of it.
  • I believe that this test can be sort of a guide to different paths of research we can take. The quest of dealing with each part of the test might lead us to actual progress in the search for an Artificial General Intelligence.
  • My evaluation doesn’t have to be practical but can be thought of as a thought experiment to understand the next steps towards these Sci-fi goals.
  • It might be an overcomplicated test, but be able to test consciousness might lead us to understand what consciousness actually is and the different applications for it. If we can find intermediate solutions to the way we can save experiences, deploy these experiences and execute them later, we are already halfway there.
  • This discussion is important if we want to imitate our brain-like activity in other machines.

Disclaimer:

  • I am not a student of consciousness. I haven’t read all the philosophers’ opinions.


  1. Self-awareness - the everyday notion of being conscious. ↩

  2. Qualia - the idea that feelings associated with sensation are somehow independent of sensory input. ↩