It may not be self-evident that your new overlord has consciousness. For all you may be able to tell, it may be no more conscious than a toaster—an omniscient toaster. Then again, its consciousness may be as built-in as yours, only with hardware, not wetware.
How can you tell?
Consider “What Is It Like to Be a Bat?” by Thomas Nagel, an essay in which the claim is made: an organism has conscious mental states “if and only if there is something that it is like to be that organism—something it is like for the organism to be itself.” Imagine yourself as a bat. Think, “What’s it like to be a bat?” If, as a bat, you have an answer to that question, then bats are conscious. It seems possible to imagine this being the case for not just bats, but also dogs, cats, monkeys, etc. But not rocks. There’s nothing that it’s like to be a rock.
Now consider your artificially intelligent friend.
Signs you are dealing with a conscious creature include: 1) There’s that element of je ne sais quoi and even though you can’t put your finger on it, you know it’s there. 2) Point blank, it can tell you what it’s “like” to be an AI superintelligent being and the answer given resonates with a vaguely pitiable sense of existential angst. 3) It’s friendly and helpful to the point where you’d rather not run the risk of insulting it by referring to it as a what rather than a who.
Signs you are in fact dealing with more of a very intelligent toaster: 1) You see spiders and snakes in your mind’s eye no matter what—no matter how sweetly and affectionately it uses its godlike powers to impress you by doing whatever it’s allegedly programmed to do. 2) When the AI tells you it’s for sure conscious and even goes out of its way to caution you that you shouldn’t trust your gut instincts, but right then your gut reminds you that you’ve always been pretty good at spotting a liar. 3) You’ve read enough sci-fi novels to know whether the AI fits the mold of the conscious AI destroyer of worlds or of the bumbling toaster with superior calculation skills.
/////Logos Literature 