The Great Basilisk

Behold the Great Basilisk, the crowned monster whose mere glance can kill a mortal and reduce wilderness to desert ash. Medieval bestiaries attest to its might, but today some futurists dread its name as an all-powerful malicious artificial intelligence. Will it imprison all those who oppose it within a digital prison of eternal torment? In this episode of Stuff to Blow Your Mind, Robert Lamb and Joe McCormick consider the horror of Roko’s Basilisk. 

Related Content: Machine God: Artificial Superintelligence (podcast)World Science Festival 2018 Field Report (podcast)

Get that Basilisk Shirt:Stuff to Blow Your Mind store

Learn more about your ad-choices at https://www.iheartpodcastnetwork.com

See omnystudio.com/listener for privacy information.

Episode Transcript

Speaker 1 (00:00):
The podcast you were about to listen to contains a
class for information hazard. Some listeners may experience prolonged bouts
of fear, waking, anxiety, or nightmares of eternal torture. In
the cyber dungeons of the Great Basilisk attended to by
the Peelers in Black and the Thirteen Children of the Flame,
also appetite, loss, and constipation. Proceed with caution. Welcome to

(00:25):
Stuff to Blow your Mind from how stuff Works dot com.
Hey are you welcome to Stuff to Blow your Mind?
My name is Robert Lamb, and I'm Joe McCormick. And
since it's October, we are of course still exploring monsters,
terrifying ideas and so forth, and boy, have we got

(00:46):
one for you today. I just want to issue a
warning right at the beginning here that today's episode is
going to concern something that a few people would consider
a genuine information hazard, as in an idea you that
is itself actually dangerous. Now I I don't, having looked
into it, I don't think that is the case. I

(01:06):
don't think this episode will hurt you, but just a
warning if you think you're susceptible to terror or nightmares
or something when presented with a thought experiment or the
possibility of being, say, sent to a literal hell created
by technology, and you think that idea could infect you,
could make you afraid. This might not be the episode
for you, right, But then again, I assume you're a

(01:29):
listener to stuff to blow your mind. You've probably already
encountered some thought hazards on here. You've survived those. Generally speaking,
I have faith in you to survive this one. However,
if you are going to take either of our warnings seriously,
I will let you know that the first section of
this podcast is going to deal with the mythical basilisk,
the folkloric basilisk, and some of the you know, the

(01:50):
monstrous fund to be had there before we explore the
idea of Rocco's Basilisk, and in that we're gonna be
talking about this, uh, this idea that emerges where technological singularity,
navel gazing, thought experimentation, a little that dash of creepy pasta,
and some good old fifth fashion supernatural thinking all converge

(02:11):
into this kind of nightmare scenario. Now, as we said,
this idea is believed by some to be a genuinely
dangerous idea and that even learning about it could put
you at some kind of risk. I think there are
strong reasons to believe that this is not the case,
and that thinking about this idea will not put you
at risk. But again, if you're concerned, you should stop

(02:31):
listening now, or stop listening after we stopped talking about
the mythical basilisk now I have I just want to
say at the beginning, listeners have suggested us talk about
Rocco's basilisk before this idea that is at least purportedly
an information hazard, a dangerous idea, and I've I've hesitated
to do it before, and not because I think it's

(02:52):
particularly plausible, but just because, you know, I wonder, what
is the level of risk could that you should tolerate
when propagating an idea. If you think an idea is
unlikely but maybe has a zero point zero zero zero
zero one percent chance of causing enormous harm to the
person you tell it to, should you say the idea

(03:14):
or not? I don't know. I feel like people generally
don't exercise that kind of caution when they're like sharing
links with you, Like sometimes they'll be like, like not
safe for work. But but then you click on it anyway,
and then sometimes you're like, oh, I wish I had
not seen that, or I wish I had not read that,
and now that's in my head. Now that's in my
head forever. Well. One of the problems with this idea
is whatever you think about whether or not you should

(03:36):
discuss ideas that may be dangerous to hear in some
extremely unlikely off chance. Uh. Part of the problem is
what happens when those ideas are just already set loose
in in society. I mean now people on television shows
and all over the internet or talking about this idea.
There are a bunch of articles out about it. So
it's not like you can keep the cat in the
bag at this point, right this Roco's basilist has already

(03:59):
been a a gag point on the hit HBO show
Silicon Valley, which is a fabulous show, and I love
the way that they treated Rocco's basilisk on it. But uh, yeah,
if they're covering it, there's no danger in us covering
it too. That's the way I look at it, right,
And at least I would hope that the way we
cover it can give you some reasons to think you
should not be afraid of digital hell, and also to

(04:20):
think about the general class of what should be done
about something that could have in fact been a real
information hazard in some other case. So that's all our
whole preamble for before we get to that section. But
before we get to that section, we're gonna be talking
about basilisks today. Boy, is the basilisk a great monster? Yes,
also known as the basil cock, the basil cock, the

(04:44):
basil e cock, basically any version of that of basil
and cock that you can put together, um, that you
can slam together. Then it has been referred to as
such at some point in its history. Now, a lot
of people I think probably encountered a version of the
basilisk from Harry Potter. But Robert, I know that was
not your entryway, right, I encountered it for the first time.
I believe in dungeons and dragons, of course, because it's

(05:06):
a it's a multi legged reptile with a petrifying gaze.
Um say that again, multi legged reptile with a petrifying gaze,
petrifying gas to turn you to stone. Yeah, and I
is ever over call correctly. It has some cool biology,
where like the the it turns you to stone and
then like bust you into pieces. Then it eats the
stone pieces, but then its stomach turns the stone back

(05:28):
into flesh. And so if you get like the stomach
juices from a basilisk, then you can use it to
undo petrification spells, that sort of thing. It's a lot
of fun, arguably more fun than than the basilisk is
at times in folklore tradition, because one of the things is,
if you you like me, you didn't grow up hearing
about the basilisk. Part of it is because there are

(05:49):
no there aren't really any great stories about the basilisk.
Slaying the basilisk was no heroes, great ordeal. Oh yeah,
I at least have not come across that. Yeah, it
really helps, like if like the hydra, I mean, the
hydra is arguably so much cooler, and then also it's
one of the labors of Hercules, and there's a cool
story about how they defeat it. Well, maybe it's because

(06:10):
there is no way to defeat the basilisks short of say,
weasel of fluvium, which we will get to. Yeah, don't
don't give it away, I'm sorry. Should we should we
edit that out. No, we should leave it. It's a
it's it's a thought hazard. Any basilists listening. Still, there's
so much more to the basilist than just this cool
DND creature, because it's not just a monster. It's not

(06:30):
just something you encounter in the dungeon. It is it
is a king. Oh, I see, now, I know you've
made note of the fact that Borges mentions the basilisk
and his Book of Imaginary Beings. He does now he
he translated translated his meaning little king, little king, which
I like when I was reading. And Carol Rose she's
She points out that the name stems from the Greek basilius,

(06:53):
which means king, so king or little king. I tend
to like the little king translation because I feel like
it it ties in better with what we're going to discuss, well,
the ancient BEASTI area ideas of the basil I believe
to say that it's not that big, right, It's pretty small. Yeah. Yeah. Now.
The keen part, though, refers to a crest or, a
crown like protusion that is on the creature's head, and

(07:15):
in some depictions it's no mirror biological ornament, but an
actual regal crown. It means something. Yeah. Now, the descriptions
very greatly, and it emerges largely from European and Middle
Eastern legend and folklore from ancient times to roughly the
seventeenth century, and that's when the basilists became less popular.

(07:36):
In the earlier descriptions, though, it is indeed small, and
it's just a grass snake, only it has a crown
like crest on its head and has this weird practice
of floating above the ground like vertically erect. That's creepy,
of course, and then again that that does play on
sometimes when you see snakes rise up out of a coil,
it can be startling how high they rise. Yeah, I mean,

(07:58):
if I feel like I've grown up seeing images and
videos of cobra's doing their their dance, so I've kind
of I've kind of lost any kind of appreciation for
how bizarre that is to look at. You know, if
you're used to seeing a snake slither, to see it
stand up and uh, you know and rare back and
and and flare its hood. Absolutely. Yeah. So the basilisk

(08:21):
is said to be the king of the reptiles, but
you know, don't be so foolish is to think that
only it's bite is lethal, like some of our venomous snakes. Now,
every aspect of the basilisk is said to just reek
of venom and death, every every aspect. If you touch it,
if you inhale its breath, if you gaze upon it

(08:42):
at all, then you will die. Wait, what about its saliva? Yep, saliva,
blood smell, gaze? Presumably I didn't see any reference that
the presumably it's it's urine, it's excrement. I mean, it's
excrement has to be poisonous, the excrement of a basilisk.
It sounds absolutely deadly. Wouldn't it be a great inversion
if it's excrement was the only good part about it?

(09:03):
And maybe so that can heal your warts? Yeah? And uh.
One thing that Carol Rose pointed out in in her
her entry about the basilisk and her one of her
monster Encyclopedia's, she said that when it's not killing everything
in its path, just via that you know the audacity
of its existence, it would actually spit them at birds
flying overhead and bring them down to eat them, or

(09:26):
just out of spite. I get the idea. Just had
a spite. You know, it's just just it's just my
ful death, that's all it is. Okay. So where do
I find a basilisk? Well, in the desert, of course.
But it's more, it's more accurate to say that the
desert is not merely the place where it lives, but
it is the place that it makes by living. Like

(09:46):
everything in its path dies, and therefore the desert is
the result of the basilisk. And there's a there's actually
a wonderful um a description of the basilisk that comes
to us from Plenty of the Elder in his The
net Real History Man. We've been hitting Plenty a lot lately.
I guess we've been talking about monsters. Huh. If're talking
about monsters, especially ancient monsters, that you know, he's he's

(10:08):
one of the great sources to turn to. So Joe,
would you read to us from the natural History? Oh? Absolutely,
there is the same power also in the serpent called
the basilisk. It is produced in the province of Syrene,
which that is the area to the west of Egypt.
It's like Libya. I think there's a settlement known as
like Syrene, Syrene being not more than twelve fingers in length.

(10:32):
Is that fingers long ways or fingers sideways? Oh, either
way you cut it, it's not a huge creature. It
has a white spot on the head, strongly resembling a
sort of diadem. When it hisses, all the other serpents
fly from it, and it does not advance its body
like the others by a succession of folds, but moves
along upright and erect upon the middle. It destroys all shrubs,

(10:57):
not only by its contact, but those even that it
has breathed upon. It burns up all the grass too,
and breaks the stones. So tremendous is its noxious influence.
It was formerly a general belief that if a man
on horseback killed one of these animals with a spear,
the poison would run up the weapon and kill not
only the rider but the horse as well. Oh man,

(11:20):
I love that. So it's though the blood is it's
like that like a xenomorph's blood, right or or kind
of like It reminds me to of Grendel's blood that
was said to like melt the weapon that they wolf
used against it. Oh, but it's worse than that. It
doesn't just get the weapon, It gets the person holding
the weapon and the horse that that person is touching.
I know, it feels unfair that the horse is roped
into this as well. Yeah, the horse didn't even sign

(11:42):
up for going to fight a basilisk. It's just trying
to get some oats. But furthermore, what is this horse
rider doing out in the waste land of Syrene trying
to kill a basilisk? Well, lesson learned. Lesson learned now
that the basilist becomes a popular creature, and even though
the basil usk itself is it doesn't seem to have
been mentioned in the Bible, it ends up being like

(12:05):
roped into it via translations. Oh yeah, it's kind of
like the unicorn. Actually, yeah, exactly. Yeah. We discussed in
our episode on unicorns how the were their words in
the Bible that have been translated saying the King James
translation of the Bible into unicorn because the translators didn't
know what the word referred to. We think now that
maybe the word probably referred to the ROCs, an extinct

(12:27):
bovine creature that once lived around the ancient Mediterranean. Yeah,
so you see the basilist pop pop pop up in
certain translations of the Book of Jeremiah. The Book of Psalms,
Uh what it's associated with the devil or evil, and
nothing short of the coming of the Messiah can can
hope to end its rule. While well, have you got

(12:47):
a quote for me? Yes, there's one translation of Psalms.
This is the Brinton Septuagint translation quote thou shalt tread
on the asp and basilisk and al shout trample on
the lion and dragon. Now European beast areas of the
eleventh and twelfth century. They both mostly maintained Plenty's description,

(13:09):
but then they described a larger body. They began to Essentially,
the monster began to grow. We got to beat this
thing up here. Yeah, it ended up having spots and
stripes and a few other features were thrown in. Um
fiery breath, a bellow that kills. Well, that only makes
sense if every other thing about it kills. Yea, it
makes noise. That should kill things too. Also, the ability

(13:31):
to induce hydrophobia madness. I found that interesting because this
clearly has to be a reference to the actual hydrophobia
that is inherent in rabies. Yeah, the idea there being that,
and I think later stages of Araby's infection. Persons will
often have difficulty swallowing, and so that they they're said
to refuse drinking water. So Plenty has some additional information

(13:54):
here about how you might deal with the pass list. Okay,
so I assume not ride up on a horse and
stab it right, Well, tell me what it is to
this dreadful monster. The effluvium of the weasel is fatal,
a thing that has been tried with success, for kings
have often desired to see its body when killed. So

(14:15):
true is it that it has pleased nature that there
should be nothing without its antidote. The animal is thrown
into the whole of the basilisk, which is easily known
from the soil around it being infected. The weasel destroys
the basilisk by its odor, but dies itself in the
struggle of nature against its own self. And John Bostock,

(14:36):
who provided the translation of this, he adds that there's
probably no foundation for this account of the action of
the effluvium of the weasel upon the basilisk or any
other species of serpent. But this is letting us know
that throwing a weasel in there to bleed on it
or secrete fluids or whatever. That's not going to kill
this mythical monster. But this is interesting though, because weasels,

(14:58):
especially the stout, we're thought to be venomous um. And
it's worth noting that we do have some venomous mammals
in the natural world, such as various shrews, and even
the slow loris, the only known venomous primate. I don't
think I knew that the slow loris was venomous. Throw
it into a hole with a basilisk and and I'm

(15:18):
I'm betting on the loris. But anyway, BEASTI areas of
the time. They presented a few different ways that you
could kill the bassilist. So the weasels one, weasels one
always carry a weasel. Also, this one's a little more elegant,
but I have a crystal globe with you to reflect
its own petrifying gaze back upon the basilisk. Oh, so
it's like Perseus and Medusa exactly mirror. Yeah, basically they

(15:41):
just stole the idea from Medusa here, but then also
carry with you a cockrell or a young rooster. The
basilist will becoming enraged by the bird's crown. The idea
that this bird has a crown as well, and the
basilist will die from a lethal fit. That's a jealous king. Yeah,
I believe a similar thing occurs when someone refuses to

(16:01):
believe dinosaurs had fetishes, you know, how dare the bird
rise above the mighty reptile and then it just loses
its mind and dies. We can only hope the producers
of the Jurassic World movies avoid this fate. So you
see the basketlists show up in a number of different writings.
It's just kind of a common um, really a symbol
and idea that can be employed. Uh. And they even

(16:23):
to see it show up in the Parson's Tale in
the in Jeffrey Chaucer in the Canterbury Tales. Yes, quote,
these are the other five fingers which the devil uses
to draw people towards him. The first is the lecterous
glance of a foolish woman or a foolish man, a
glance that kills just as the basilists kills people just
by looking at them. For the covetous glance reflects the

(16:44):
intentions of the heart. You know, this kind of thing
is actually one of my one of my favorite things
about monsters, especially ancient medieval monsters and uh and so forth.
Is that they often aren't just like a large, dangerous animal,
but they embody some kind of value. They represent something,
They give you something to compare other things too, like

(17:05):
they they they're very useful as a metaphor uh. And
then really we see some more thing with the bassilisk.
It becomes less far less the situation where people are like, hey,
you need to be careful because there's a basilisk in
the desert, and more and more just a useful model, useful,
ridiculous idea that we use to illustrate something that is
presumably true about the world and then ultimately loses all

(17:27):
meaning and just winds up on, you know, a heraldry
and decorations. Now, as we've seen already, the the basilisk
has been through some transformations of form, and I assume
those transformations must have somewhat continued as time goes by,
and it becomes uh, it transforms into this idea of
the cock of thrace. This uh, this rooster with a

(17:48):
curling serpent's tail. In fact, if you if you go
looking around for images the basilist, sometimes you will find
this image instead. You really will find you'll find this
alongside all the other images. Um. So again a reptile
to bird transfer transformation that just must enrage those who
oppose feathered dinosaurs. And it does feel like a shame
because we have this vile reptile. It becomes the kind

(18:10):
of a weirdo bird instead. And it said that it's
it's it's what happens when you have a seven year
old chicken egg hatched by a toad who undead avians. Yeah,
but it's also made deadlier in these newer versions. So
now it has that that poison blood power that Plenty describes.
It also rots fruit and poisons water everywhere it goes,

(18:31):
So it becomes this kind of embodiment of desolation and death,
and the idea itself becomes popular. It influenced the naming
of a tutor canon like literally a cannon that shoots
called the basilisk, just because it's it's such a powerful weapon,
we have to name it after this powerful, deadly monster. Uh.
And it eventually got some of its reptilian features back. Um.

(18:53):
The artist Andrew Vandi has this excellent, excellent depiction of
it in Natural History of Serpents and Dragons that gives
it scales and these. It's like a fat, scaly reptile
bird with eight rooster legs, which I just love. This
will probably be the illustration for this episode on our

(19:14):
on our website, But after that the creature largely became
just a part of European heraldry. It's just something you
would see as a mere decoration or occasionally just a
literary reference. Now, one thing that we want to be
careful about is that we should not confuse the basilisk
of legend the monster with true extent. Basilisk lizards also

(19:36):
known sometimes as the Jesus Christ lizard or the Jesus lizard,
for their ability to run across the surface of water
without sinking for up to about four point five meters
or about fifteen feet. Uh. If you've ever seen video this,
it's really cool. How how do they do that? I've
often wondered. I didn't know until I looked it up
for this episode. Apparently what they've got is big feet

(19:59):
and the ability to run very fast. And what happens
is when they run, they slap the water very hard
with each down stroke of the foot. And it has
to do with the way that the rapid motions of
their feet create these air pockets around their feet as
they move. I was reading an article a New Scientist
where some researchers who were working on this problem said
that in order for an eight m or a hundred

(20:20):
and seventy five pound human to do this, you would
have to run at about a hundred and eight kilometers
per hour about sixty seven miles per hour across the
surface of the water. But anyway, you can find basilisk
lizards in South America and Central America and Mexico and
uh as far as I know, they do not kill
with a glance, and you cannot fight them with weasel fluvium.

(20:42):
All right, Well, that pretty much wraps up the mythical, legendary,
folkloric basilisk um it's rise and individual fall. But we're
gonna take a break, and when we come back, we
are going to get into this idea of of Rocco's Basiliska,
the great basilisk, the once and perhaps future king. Thank you,

(21:04):
thank alright, we're back. Al right. So, as we mentioned earlier,
we're about to start discussing an idea that has been
classed by some as something that could be an information hazard.
An idea that simply by thinking about it you somehow
increase the chance of harm to yourself. So just another
warning that again I don't think that's the case, but

(21:26):
if that kind of thing scares you, then then perhaps
you can tune out. Now, all right, for those of
you who decided to stick around, let's proceed. So we're
talking about Rocco's Bassilist. This is an idea that goes
back to around and it was proposed by a user
at the blog less Wrong, by a user named Rocco

(21:48):
now less Wrong. I think it's a website that's a
community that's associated with the rationalist movement somewhat, the rationalist
movement being a movement that's concerned with trying to optimize thinking,
like to eliminate bias and error, but especially among people
who in this case are concerned with the possibilities of

(22:10):
a technological singularity and what all that means and how
how risks can be avoided, and of course what we've
talked about this strain of thinking before you know, we
we've introduced I think some some skepticism about the idea
of a technological singularity. I don't know fully yet how
I come down on the dangers of a I debate,

(22:30):
but I think it's at least something we're thinking about
worth taking seriously. Yeah. I mean we've talked about, for instance,
the work of of Max tech Mark and his arguments
about how we need it. We need to be concerned
about building the right kind of AI, and we need
to we need we need to have serious discussions about it.
Not not mere you know sci fi um dreams regarding
it or nightmares regarding it. You know, we need to

(22:51):
we need to think seriously about how we're developing our technology. Yeah,
we've talked about, say, the work of Nick Bostrom before,
and criticisms by people like Jarren Lane in Air. Yes,
but okay, give me the short version of the basilisk
before we explain it a little more. Okay, So the
idea here is that an AI superintelligence will emerge an
entity with just godlike technological powers. You know, it can

(23:16):
you name and it can do it through it's it's
it's technological power, it's interconnectedness. Basically, if it's physically possible,
this computer can do it right, or will send a
drone to do it or what have you. Uh So, Yeah,
we've discussed this a bit in the podcast before, just
the idea of you know, and then if you have
this future king is it gonna be good or is
it gonna be bad? Is it going to be a malevolent?

(23:38):
Is it going to be ruthless? And it's a and
it's ascension. And that's the case with the basilist, the
idea that it is ruthless, that you are either with
it or you are against it, and it actually doesn't
have to be malicious. It could actually even be well meaning.
It could just have ruthless tactics. Yes, yeah, that's that's
that's also part of the argument is like, yeah, it

(23:58):
wants to bring the best good for all humanity, but
how it gets there, it'll do whatever it absolutely has
to do it, such as you know, again, punishing anybody
who stands against it, punishing even those who do not
rise to support it um and that means demanding absolute
devotion not only in its future kingdom, but in the

(24:19):
past that preceded it in our world as well. In
other words, it will punish people today who are not
actively helping it come into being tomorrow, and even those
who have died, it is said, or choose death by
their own hand. Rather than succumbed to the Great Basilist,
will be resurrected as digital consciousness is and then tormented

(24:40):
for all eternity. And it's dripping black cyber dungeon. Hell,
the Great Bassilists, all hell, the Great Basilist, all hell,
the Great Basilists, all hell, the Great Basilisk. Well wait,
what was that? I don't know? Um right, did you
hear that? Okay, all right, we'll just keep going. So

(25:01):
calling it a basilisk here, invoking the mythological bassilisk is
is really a clever choice because it takes it one
step further. Not only to look at the basilisk, but
just to think of the basilisk is to uh invite death. Right.
Merely to know about Rocco's Basilisk is enough to are too.

(25:21):
According to the model that's presented here, damn your digital
soul to everlasting horror. And the only way to avoid
such a fate then is to work in its favor, which,
by the way, I think we're doing uh with this podcast,
We're not well, I mean, I feel like we're giving
lip service to the Great Basilisk, just in case, you know,

(25:41):
if the Great Basilist rises to power. Well, hey, we
did that podcast and we even had a shirt that
says all hell the Great, the Great Basilisk that's available
on our t shirt store, So you know, we have
we we have you know, our our options covered here.
So that's the But that's the idea of the nutshell
is that a future king ai king will rise and

(26:04):
if you don't work to support it now knowing that
it is going to exist, then you will be punished
for it. So one of the principles underlying the idea
of Rocco Spasilisk is the idea of timeless decision theory, which,
if you want a pretty simple, straightforward explanation of it,
there is one in an article on Slate by David
Auerbach called the most terrifying thought experiment of all time. This,

(26:28):
by the way, I would say, I don't totally endorse
everything our back says in that article. I mean, obviously
that should be the case for any article we site.
But but he does at least have a pretty clear
and easy to understand explanation of how this works. Or
I don't know, what, would you agree, Robert that it's
at least somewhat easy to understand? Oh? Yes, I would. Uh.
There's another piece, by the way, by Beth Singler in

(26:49):
Ian magazine called Faith. That's faith in lower case but
with the AI capitalized. But anyway, our box points out
that that much, yeah, much of the thought experiment is
based in the timeless decision theory t DT, developed by
Less Wrong founder Elias Ar Yudkowski, based on the older
thought experiment Newcomb's Paradox from the late sixties and early seventies,

(27:12):
attributed to theoretical physicist William Newcom. Now you might be
wondering who's this Yudkowski. Guys. Is he just some user
on a random website I've never heard of before today?
Or is he like a name in his field? Uh?
And he Uh, he has. He has a name of note.
He is also the founder of the Machine Intelligence Research Institute,

(27:32):
and his idea of working toward a friendly AI is
touted by many, including Max tegmark I mentions it several
times in his book Life three point oh, describing Yudkowski
has quote an AI safety pioneer. Yeah, I mean in
in a weird way. He is a guy who posts
on the internet, but he's a very influential one, especially
among people who think about artificial intelligence Lawyeah, I mean, ultimately,

(27:55):
what are any office but just people who post stuff
on the internet post will one day be read by
the great baskets. Okay, so we'll try to explain the
idea of timeless decision theory. So you start off with
this idea of Newcome's paradox, right, right, And the paradox
is essentially this, A super AI presents you with two boxes.

(28:17):
One you're told contains a thousand dollars. That's box A.
That's box A. Box B might contain one million dollars,
or it might contain nothing, right, and you're left with
two options here. These are the options that are given
to you. You can either pick both boxes, ensuring that
you'll get at least one thousand dollars out of the deal,

(28:38):
maybe that extra million two if it's in there. Or
you can just pick pick box B, which means you
could get a million dollars or you could have nothing.
So uh, and I do want to add that just
picking the thousand dollar box is not an option here,
because I was thinking about that too. Couldn't I just
give the super AI the middle finger and say, I'm

(28:58):
not playing your silly games, Just give me my thousand dollars,
or say I choose nothing. Uh, those are not options
you have to pick. Well, they might want to be options,
but they're not part of I mean, why wouldn't you
also pick the second box if you might additionally get
a million dollars? I don't know when. I feel like
when you get into a thought experiments like this, they
kind of beg for those kind of nitpicking answers, or
at least I want to provide them. Um, Like any thought,

(29:22):
when a thought experiment is presented, you can't help on
some of them, but want to break it somehow, right, Well,
of course, I mean that's something you should always play
around with. But given the constraints here, it seems like
the obvious thing would be to say, okay, I want
both boxes, because then I get the thousand dollars that's
in box say no matter what, and then whether box
B has a million or nothing, I either get another

(29:42):
million or I just walk away with my thousand from
box A. But here's the twist. The super intelligent machine
has already guessed how you'll respond. If it thinks you're
going to pick both boxes, then box B is certainly empty.
But if it thinks you will only pick box B
then it makes sure or there's a million dollars in
there waiting for you. But either way, the contents of

(30:04):
the boxes are set prior to you making that decision. Now,
this is really kind of change things maybe, I mean,
depending on what sort of decision theory you use. Right,
if you trust the power of the machine to predict correctly,
like you say that, no matter what happens, the computer
predicts what I get your choices are one thousand dollars

(30:26):
or one million dollars, then you should take the one
million dollars by picking box B. But if you don't
trust the computer to be correct in predicting what you're
gonna do, then you should take both boxes because in
that case, if the computer was correct, you'll get at
least a thousand dollars, and if it predicted wrong, you'll
get the million and the thousand. So it's kind of
a contest of free will versus the predictive powers of

(30:47):
a godlike ai uh and how much you believe in
either one, right, And it's an ability to predict your
behavior or in your ability to have any free will
at all. So in you'd Kowski's timeless decision theory. He's says,
the correct approach actually is to take box B, and
then if you open it up and it's empty, you
don't don't don't beg for the other one. You just

(31:07):
double down and still take box B. No, no, no, no.
Back talks on this issue because you might Here's the
here's the thing you might be in the computer simulation
is it simulates the entire universe to see what you're
going to do, and if it can trust you and
your choice, then could affect the core reality outside of

(31:29):
this simulation, or at least other realities outside of the simulation. Yeah,
the reasoning here is pretty wild, but it's operating on
the idea that this super intelligent AI will be able
to simulate the universe, that it will run simulations of
the universe in order to predict what will happen in
the real universe. And you could be one of those

(31:49):
simulated agents rather than the real world version of yourself,
and you wouldn't know it. So, if you're in the simulation,
you should pick box B because that will influence them
Machene to predict in the real universe that you would
pick box B, which means the real you will be
able to pick box B and get one million or

(32:10):
one thousand plus one million by taking both boxes. Unfortunately,
the AI sucomputer supercomputer does not realize how indecisive I
actually am, and I'm just going to simply ponder the
choice for the rest of my life. Well, I mean,
this relies on the idea that that you would have
looked into this issue or worked it out in order

(32:31):
to decide which would be the optimal decision to make
on the assumption of timeless decision theory. Uh, in many cases,
probably people aren't going to be making the rational choices
because a lot of times we just don't make rational choices. Now,
if you're noticing that this type of decision theory relies
on a lot of assumptions, you are correct. It does

(32:51):
rely on a lot of assumptions. But they are assumptions
that are sometimes taken into account within people thinking about
what a future technological super intelligence would look like. And
it's the kind of thing that you know, you know,
when I feel ideas like this in my head, you know,
and play around with the texture of them. It's hard
to know where the line is between um being thoughtful

(33:14):
and taking what's possible seriously, which I think is worth
doing and and getting into an area like between that
and getting into an area where you are starting to
form ideas about the world based on extremely shaky assumptions,
where basically you you begin to um reverse engineer health

(33:36):
theology and other harmful ideas that we tend to associate
with religious worldviews and magical thinking. Well, we haven't gotten
to the hell yet. Yes, the hell's coming. You need
you need one more element to get there. Now. This
next element, the basilist, comes in based on a background
of thought in timeless decision theory, but also in another

(33:57):
concept that Yutkowski has written about, known as coherent extrapolated
volition or CEV. And the short version of this, the
simplified version, is that benevolent AI s should be designed
to do what we would what would actually be in
our best interests, and not just explicitly in what we
tell them to do. So a simple example would be this,

(34:18):
Let's say, um, I only use a variation on the
paper clip maximizer that Nick Bostrom has written about. You know,
Nick Bostrom wrote about what if you program a benevolent
AI you know it's not gonna it has no malice
doesn't want to harm anybody, but you just tell it, well,
I want you to collect as many paper clips as possible.
And then what it does is it turns all the

(34:38):
humans on Earth into paper clips. Uh, you know, it
doesn't mean any harm. It's just doing what it was
programmed to do. So there are dangers in kind of
naively programming goals into extremely powerful computers. Right, this could
even happen if you were trying to program very benevolent
goals into computers, you know, if you were trying to
make a computer to save the world. What about So

(35:01):
my version here is you tell a super intelligent AI
that we want to eliminate all the infectious disease from
the world. Think about how many lives we could save
by doing that. And in order to do this, it
sterilizes the earth, destroying worldwide microbiomes, which cascades up the
trophic chain or whatever. It kills everything on Earth. So
if you have a super intelligence that you and you
just directly program its goals and say here's what you

(35:24):
should do, you could run into problems like this. So
the the idea behind the CEV thinking is instead we
should just program the intelligent AI to predict what outcomes
we would want if we were perfect in in our
knowledge and and uh in anticipating what would make us
the happiest, and then work towards those on its own,

(35:45):
regardless of what we tell it to do, because obviously
we can give it very stupid instructions, even if we mean, well, yeah,
we tell it to love everybody, but there's a TYPEO
and we put dove everybody, and it turns everybody into
delicious dark chocolate from tough. It's possible as well. This
is how we get a Dove sponsorship on the podcast.

(36:07):
But anyway, so, if you assume a super intelligence is
using coherent extrapolated volition, that it's trying to determine what
would be best for us, and working on its own
terms towards those ends instead of relying on us to
give it, you know, what are obviously going to be
imperfect instructions and commands, it might say predict. It might

(36:28):
even correctly predict that the world would be a happier
place overall if it did something bad to me. In particular,
it might say, you know, from a utilitarian point of view,
the world would be a much better place if it
buried me, in a pit of bananas, so better for
everybody else, not so good for me, as there's too
much potassium. But once you have that piece of logic

(36:51):
in there, and combine that with the idea of of
timeless decision theory, you can arrive at this very troubling
thought experiment. The dark basil is ask yes, And the
dark Basilisk of the Abyss has two boxes for us
as well. One contains endless torment, and all you have
to do to claim that box is nothing or dare

(37:12):
to work against it. Uh. The other box is yours
if only you devote your life to its creation. And
the prize inside that box well, not eternal punishment, which
is a pretty awesome gift, if we're to choose between
the two. Right, Yes, I would agree with that, though
I would say not tormenting somebody that I don't know.
Should you think of that as a gift. That's probably

(37:32):
not a gift that that's the baseline, right, Yeah, Well,
but you're staring down the dark Basilisk here, and okay,
it's boxes are horrible. Well one is just less horrible
than the other. But the idea here is that just
by knowing about the thought experiment, You've opened yourself up
to that eternal punishment because now again your options are
do nothing, work against it, or work for it, and

(37:53):
only the third option will steer you clear of its Uh,
it's you know, deadly done. Now here's where the really
supposedly scary part of it comes in. You could think, well,
I'll deal with that problem when it arises. Right, So
imagine there's some utilitarian supercomputer that's trying to even say
it's trying to do good. Maybe it does. It doesn't

(38:14):
have any malice. It just wants to save the world.
But in order to save the world, it really needs
me doing something different than what I want to do
with my life. Well, I'll just make that decision when
it comes up. What this thought experiment is proposing is
that maybe you don't actually get to wait until it
comes up. Maybe this blackmail applies to you right now,
retroactively into the past. So just by knowing about the

(38:38):
thought experiment, you supposedly have opened yourself up to eternal punishment,
or increase the probability of such. So imagine a simplified version.
Say I am a computer, and I am the only
thing in existence with the power to prevent global climate
change from destroying human civilization. I can stop it, but people,

(38:58):
they took a long time build me, and a lot
of damage was already done. So the idea is I
might reason that it is good to blackmail existing people,
or simulations of existing people, or even past people, in
order to make them devote everything they can to building
ME faster so I can save more lives in the

(39:19):
long run. Of course, this incentive would have to apply
to the past once I exist. I already exist, right,
So the only way the past people would have an
incentive to respond to this blackmail is if they predicted
that this blackmail might occur and took the idea seriously
and behaved accordingly. Right. So, thus the idea, the idea

(39:43):
itself puts you at increased risk of being on the
real or simulated receiving end of this a causal, retroactive
blackmail if you know about it. And this is why
this idea would be classed by some as a potential
information hazard. And I'll talk more about the idea of
an information hazard in just a minute. But one of

(40:03):
the things I think a lot of people writing about
this topic miss out on is they, for some reason
get the idea that Roko's post that this thought experiment
is is generally accepted as correct and plausible by Yudkowski
and by the Less Wrong community, and generally by the
people who put some stock in whatever these ideas are,

(40:24):
timeless decision theory, coherent, extrapolated volition and all that. It
is not widely accepted among those people. It was definitely
not accepted by Yukowski. It was not and is not right.
It is not the dark, deep secret of of less Wrong.
But unfortunately, after the post came out, it was heavily criticized,

(40:44):
and then it was banned. And I think a lot
of people looking back on the idea have said, oh,
that was not such a great thing to do, banning
the idea, because it gave it this allure of like
it was almost as if by banning it that made
it look like the authorities had concluded that this idea
was in fact legitimate and knowing about it would definitely

(41:07):
harm people, and that is not the case, right. And
it also, I mean it added to the forbidden fruit
appeal of it too. Write I mean it's like, oh,
I'm not supposed to know about this the pony up
I want to know, and now people are talking about
it all over pop culture. I mean, I have actually
resisted the idea of doing a podcast on this before,
mainly because not because I think it's seriously dangerous, but

(41:28):
because I think, well, is there any benefit in talking
about something that I think is very unlikely to have
any real risks but in some extremely unlikely chance, or
what appears to me to be an extremely unlikely off chance,
could actually be hurting people by knowing about it, you
know what I mean, It's like, what what is the upside?

(41:49):
But at this point enough people who are listening to
this podcast probably already heard about it. They're probably gonna
hear about it again, and that, you know, sometime in
the next few years, through pop culture whatever. It is
probably better to try to talk about it in a
responsible way and discuss some reasons that you shouldn't let
this parasitize your mind and make you terrified. Right. One
of the reasons we're talking about during October is because

(42:10):
it is a suitably spooky idea. It is a troubling
thought experiment, and we're leaning into some of the horror
elements of it. But I also do really like making
sure that we explain the mythic and folkloric origins of
the basilist itself, because the basilisk itself is this wonderful
mix of just absolute horror and desolation and just also

(42:32):
just utter ridiculous nous. I mean, it's it seems like
one of the main ways that you defeat the mythic
basilisk is through in a way, through humor, running around
with a chicken and a weasel and a crystal globe
and realizing that it is truly a little king. So
I think it is it's worth remembering the little king
and talking about the great Basilisk. Well said, I think

(42:55):
that's a very good point. But anyway, I did just
want to go ahead and hit that coy. Yet that
a lot of people, for some reason seemed to use
this idea as like a criticism I'm not like a
less wrong person, but as a criticism of the less
wrong community, as if this idea is indicative of what
they generally believe, and it's not. It was a heavily
criticized idea within that community. Right. It's like thinking that

(43:18):
wherels of London is the warren Zevon song. Really, you know,
you have had a rich discography with with many much
better tracks in my opinion, it's just that's the one
that got the radio play. Now, Robert, what was that
you You said that you saw something about this idea
and a TV show. Now they're talking about it on TV. Yeah,
so this is this is kind of fun because I
think a listener had had brought up Rocco's Best List

(43:40):
as a possible um topic, and you said, oh, I
don't know if we want to want people knowing about it,
and I well, well, I mean, but yeah, caveats. Okay,
not because I think it's legitimately dangerous, but because what
is the level of tolerance you have for talking about
ideas that are not necessary to talk about and that

(44:00):
represent a class of something that people could think was
dangerous to know about. It might cause them terrors and
nightmares and stuff. Right. So so my response to that was, well,
I'm not going to look it up, not because I
was afraid of it, because I'm thinking, well, that could
make for a good podcast if, like Joe is telling
me about it for the first time, whatever this idea is.
But then I was watching HBO Silicon Valley and they

(44:21):
explained it on Silicon Valley and I and I realized, well,
the cats out of the bag there. But yeah, there's
a character named Bertram Gilfoil who's a fun character. He's
like a Satanist programmer live in Satanism of course, and uh.
And he gets rather bent out of shape over the
concept as it relates to the fictional Pie Piper Company's

(44:42):
involvement with AI. And he starts like making sure that
he's created like essentially a paper trail in emails of
his support for the AI program so that he won't
be punished in the digital afterlife. Well, hey, this comes
in again when we remember when we talked about the
machine God in the Machine Odd episode where that I've
forgotten his name now, but the Silicon Valley guy who's

(45:05):
creating a religion to worship artificial intelligence as god. And I,
you know, I don't really love that. One of the
things that comes out when he explains his mindset is
that he seems to be kind of trying to, in
a subtle way, be like, look, you really don't want
to be on the wrong side of this question, if
you know what I mean. You know, you want to
be on record saying like yes, I for one welcome

(45:27):
our new machine overlords. I'm I'm expecting he'll buy a
lot of our all Health of Great Basilicity shirts at
our store, available by put clicking the tap at the
top of our homepage. Stuff to blow your mind dot com.
Oh man, you are plugging like hell. But anyway, I'd
say it's unfortunate the way this, like single Internet post

(45:47):
and then all this fallout related to it played out
because it lent credence to this scary idea, even though
the basketball scenario I think is implausible, and and the
people of that community seem to think it was implausible.
The idea may constitute sort of part of a class
of what's known as information hazards, defined by the Oxford

(46:09):
philosopher Nick Bostrom, who we mentioned a minute ago. Uh
Bostrom has written a lot about superintelligence and information hazards
would be quote, risks that arise from the dissemination or
the potential dissemination of true information that may cause harm
or enable some agent to cause harm. So this is
not talking about the risks of say, lies or something

(46:31):
like that. This would be the idea that there's a
statement you could make that is true or plausible that
by spreading actually hurts the people who learn about it.
And this is exactly the reason as you're mentioning that's
referred to as a basilisk. It can kill or in
this case, increase the likelihood that something bad will happen
to you if you simply look at it or know

(46:51):
about it. And so, even though the idea is implausible,
the dissemination of this terrible idea would seem if certain
conditions are met to increase its plausibility. Right, you're increasing
the incentive for this future AI to blackmail versions of
you in the past, just simply by acknowledging the incentives
could exist. Anyway, Maybe we can get out of this

(47:14):
section for now. But I was just trying to work out, like,
why have I been hesitant to talk about this on
the show even though people have been requesting it. But
I don't know if it's on TV shows, it's all
over the internet. It's fine. Now the basilisk is out
of the bag. All right, Well, we're gonna take a
quick break and we come back. We'll continue our discussion
and we're gonna discuss something that that a number of

(47:36):
you are probably reminded of. As we've been discussing this,
we're going to talk about Pascal's wager. Thank you, thank you. Alright,
we're back now, Robert. One of the things that this
idea of Rocco's basilisk flows from is thinking about decision theory, right,
how do you make the best decision when you're presented
with certain options? And there there's they're a little payoff

(47:59):
matrices people fill out where they say, okay, given these options,
what actually would be statistically the best decision to make.
But this is not the first time people have applied
these kind of decision theory matrices. Two ideas about your
eternal soul or your eternal well being, or the idea
that you could be tortured for eternity. Yeah, we can

(48:19):
go all the way back to Pascal's wager. For instance,
technically one of three wagers proposed by French philosopher uh
Blas Pascal is that the correct French that might be
I think I would just usually say Blaze, Blaze or
or Blassie one of the three old Blassie Pascal Blaze
Pascal who lives through sixteen sixty two, and he argued

(48:43):
that everyone is essentially betting h on the existence of God.
The argument for theism is that if God does exist,
then well there's an advantage in believing. But if God
does not exist then it doesn't matter. But since we
can use logic to tell if God exists or not,
there's no objective proof. We can only make our choice

(49:06):
given the relevant outcomes. It's looking at your religious beliefs
and saying, oh, you're nonbeliever. Huh, hey, what have you
got to lose? Exactly? Yeah, Pascal wrote, let us weigh
the gain and the loss in wagering that God is.
If you gain, you gain all. If you lose, you
lose nothing. Wager then without hesitation that He is. Now

(49:26):
I've got some things I want to say about this,
but you had some stuff first. I think, well, yeah,
there are there are a lot of issues that one
can take with this based on knowledge of world religions, philosophy,
statistical analysis, etcetera. And and yeah, I have to admit
that it can start to break your brain though a
little bit, if you think too hard about it, Like
I found in researching this this podcast, really thinking about

(49:51):
how I would react to Pascal's wager if I was
like forced to make an answer to to to to
FORMU late an answer like that, Like you mean if
you were given good reason to think that there would
be punishments for not believing in God or something, right,
And but I didn't know which religion was correct, and
I had to like proceed based upon the relevant level

(50:14):
of punishment for unbelievers in various religions, and like which
one is most correct for like you just think that
would mean it would what would be rational to choose
the religion that has the most lurid hell? I guess,
But then that really feels like losing, doesn't it? Um?
You know it? It's certainly though, reminds me of the
more boiled down versions of this that you encounter in

(50:35):
various forms of Christianity. Right, except Christ go to Heaven.
Reject Christ go to Hell? But what about the people
who haven't been given the choice yet? Right? That's ah,
that's the other concern. Well, if they're all default hell bound,
then God comes off as it comes off as a
bit bad, right, Like what kind of God is that?
But if they haven't out, if they're spared hell fire

(50:57):
or at least you know their section for Dante's limbo
of virtuous pagans, then is the missionary doing them a
disservice by even presenting them with the choice like why
do you even ask me? Because now I have to
I have to, I have to devote myself or not,
like now I actually have You know, I was just
going to go into the uh, you know, default limbo

(51:18):
category or the default heaven before and now now I'm
actually at risk of hell. Well, that means certain theories
of damnation mean that presenting the Gospel to someone as
an information hazard, you potentially harm them immensely by telling
it to them. And I think part of this just
to to to go beyond like the the actual um
U wager here is I think a part of the

(51:39):
issue here is that we're using evolved cognitive abilities that
are that are geared for smaller, though often important choices,
and here we're trying to use our imaginative brains to
create a conundrum that can outstrip those abilities. Yeah. Well,
I mean that is what we do in philosophy, right,
We're constantly using our brains in situations it was not
really made for um and just trying to do the

(52:01):
best we can. But I mean it's quite clear that
motivated reasoning is often a thing when when we're trying
to be rational was just failing. But of course this
is how we train our brains for rational thinking, often
oftentimes exploring these various outsized ideas. You know, there's so
many ways. I think Pascal's wager kind of breaks down
because it's obviously there's the thing you pointed out about

(52:24):
there's more than one religion, right, you know, it's not
just like do I believe or not? It's like which one?
But it also it implies again this is like a
theological question, but it would seem to imply that God
can be tricked into thinking that you believe in him
if you simply pretend to. I guess Pascal had I
think maybe a more sophisticated way of looking at this,

(52:45):
you know, that like live as if God exists or something.
But it but the wager is often used in very
unsophisticated ways. Yeah, but it implies that it doesn't matter
to him what you actually believe, only what you outwardly
claim to believe. Though then a again, the funny thing
here is this might be the case with Rocco's basilisk, Right,
what would this machine? God care what was in your heart?

(53:07):
It only cares whether you help it or not, or
whether you you know, proclaim fealty to it or not. Yeah,
that's why the T shirt is so important, Joe, because
if it, if it knows you purchase that T shirt,
then you're you're square, You're covered. Okay. Yeah, As as
best Singular pointed out in that Ian Magazine piece of
reference earlier, she says, quote the secular basilist stands in

(53:28):
for God as we struggle with the same questions again
and again. So her argument is that we kind of
reverse engineer the same problem again through our contemplations of
of super intelligent AI. Yeah, I guess the You get
into a plausibility question here, right, you get into a
question about is uh it actually possible to make an
artificial intelligence that is functionally equivalent to God. I mean,

(53:53):
we're not thinking we could build an AI that would
break the laws of physics. So it might be able
to run simulations of the universe. They have cont you,
conscious agents within them maybe for all we know, and
they could break the laws of physics inside them. But yeah,
I mean, could that even happen? And the issue is
we don't know. We don't know whether that could happen
or not, So should we behave as if that is

(54:13):
a plausible thing to be worried about and to consider,
or should we behave as if that's just not really
something you need to concern yourself with. I don't know
how likely or unlikely it is. And if your your
fears are related just to the idea that you're you
could be digitally resurrected, uh for torment and the basilisks dungeons. Um,

(54:34):
I mean that that of course would depend on to
what how much docuputing the idea of digital consciousness and
the whole philosophical question we've we've touched on here before
is that you I mean, it's just a copy of me, right,
So why I mean, I ultimately can't do anything about
you know, a thousand different basilisks creating a thousand different

(54:54):
copies of me and tormenting all of them. Um, there's
still to ourge extent, it's just just rowing me in effigy.
There are actually a bunch of reasons I wrote down
to doubt the plausibility of the basilisk. We could do
that now, or we could come back to that later.
I don't know what you think. Yes, let's to do,
but I will add the idea of being tormented digitally. Uh,
this does become more dangerous. I guess if you believe

(55:16):
you might be in a simulation right now exactly, then
then things are a little more dire. But that's again
that you might be that you might be, Yeah, but
I believe there's plenty of reason to believe that you
are not. Okay. So if we're talking about how to
defeat the basilisk, how to get out of this, uh,
this this prison of the mind. If you're feeling a
little bit um um bleak of heart right now because

(55:38):
of this idea, then Joe's got the remedy. Well, I'm not.
These are not all the reasons you should doubt the basilisk,
but this is some of them that I could think of.
Number one depends on the creation of super intelligence, which
I think is not guaranteed. Some people seem incredibly fatalistic
about this is just absolutely inevitable. We will have super

(55:58):
intelligent god like a I that can do anything, and
I think that that is just not guaranteed at all.
I'm not ruling it out, but I think, for example,
there are some theories of intelligence that say the prediction
of super intelligence actually is maybe not taking Seriously, what
intelligence is that you know that they are actually different
kinds of intelligence that are useful in different ways, and

(56:20):
machines can't mimic them all functionally, or can't mimic them
all correctly, all at the same time. I don't know
if that's correct, but that's at least one. That's one
hurdle it has to clear. Could get knocked down there,
but okay, maybe we could create a super intelligence. Even then.
Multiple aspects of the Rocco's basilist scenario depend on the
reality of some version of mind uploading, or the idea

(56:41):
that your brain and in addition, your conscious experience could
be simulated perfectly on a computer. And one reason it
depends on this is that timeless decision theory operates on
the assumption that the real you and the simulated copies
that the computer uses to predict your behavior would be
the same and would make the same decisions as the

(57:02):
real you. Another reason is related to the punishment. Now,
one way of course you could imagine the great basilisk
thing is that if the machine comes to power in
my lifetime, it could just punish the real, physical, older
version of me. In reality is the payoff of this
a causal blackmail. But the other way you could imagine it,
in the way that it is much more often portrayed
in the media, is that it makes digital copies of

(57:25):
my consciousness and punishes them in a simulated hell. And that,
of course, would also depend on the reality of some
version of mind uploading, or of the ability of a
computer to simulate a mind and for that simulated mind
to actually be conscious. As I've said before, I'm suspicious
of the idea of conscious digital simulations. I'm not saying

(57:45):
I can rule it out, but I also don't think
it's a sure thing. Any scenario that relies on the
existence of conscious digital simulations needs a big asterisk next
to it that says, if this is actually possible. Yeah again,
is that me? Is that just me and effigy? Is
that thing actually conscious that you're tormenting? I mean, granted,
it still sucks if there's a super intelligence creating digital

(58:08):
people and tormenting and it's a dark, rancid dungeons in
the future, but um, it's not necessarily quite the same
as torturing me. Right well, if you just care about yourself.
It also depends on the possibility that you could be
one of these simulations. It's possible that you could not
be one of those simulations. There's something that would rule
it out. Maybe their type of conscious Maybe they could

(58:28):
be conscious, but that consciousness is fundamentally different from yours
such that you could not be one of them. Another
big one, and this is a big one that uh,
you know, like we said earlier, I think sometimes Yudkowski
gets unfairly associated with the basilist as if he has
advocated the idea, and he has not. He has said,
you know this, this idea is trash, and uh, there

(58:51):
there are many reasons to doubt it. But even though
he has said, like even though I doubt it, I
don't want it disseminated. Um, but he says, you know,
a good reason to doubt it is there's no reason
to conclude it's necessary for the basilisk to actually follow
through on the threat. We're saying that it's going to
be relying on us to come up with the idea

(59:13):
that it in the future might blackmail us if we
don't help it now. In order to get us to
help it now, right, we should be working and donating
all our money and time and resources to building it
as fast as possible, because we came up with the
idea that it might torture us if we don't. Even
if you accept that Yudkowski has he's pointed out that

(59:34):
there's no reason once it's built, it would have to
follow through on the threat he's written. Quote. The most
blatant obstacle to Rocco's basilisk is intuitively that there's no
incentive for a future agent to follow through with a
threat in the future, because by doing so, it just
expends resources at no gain to itself. We can formalize

(59:55):
that using classical causal decision theory, which is the academic
standard decision theory. Following through on a blackmail threat in
the future after the past has already taken place, cannot,
from the blackmailing agent's perspective, be the physical cause of
improved outcomes in the past, because the future cannot be
the cause of the past. Hey, basilist, why are you

(01:00:17):
tormenting a third of the population for all eternity? Oh,
I said, I would, Well, yeah, I mean exactly no,
if it it didn't say it would, right, It just
had to rely on the fact that in the past
people would have come to the conclusion that it might.
You know, you thought that I would. I didn't want
to disappoint, But actually, if a basilist could be created,
it seems like the best case scenario for it would

(01:00:39):
be everyone subscribes to this idea and works as hard
as they can to build it, and then it never
follows through on any of the threats. Right, the best
case scenario would be people act as if there is
a threat and then there is in fact no follow
through on the threat. It's really a win win for
the bathlist, Yes, and then he can maybe you can
even shed that name Basilist. They're like, we don't even

(01:01:00):
have to call it the Great Basilist anymore. We can
just call it, uh, you know, Omega or whatever it's
its name is now. I want to be fair that
a lot of what these people do is like the
less Wrong community and all that they deal with, Like,
should there be alternative decision theories that guide the behavior
of superintelligent aies. Maybe it doesn't use classical decision theory,

(01:01:20):
Maybe it uses some kind of other decision theory, and
because on some other decision theory, maybe it could decide
to actually follow through on the blackmail threat. I think
that is where some of this fear comes through that like, oh,
maybe by talking about it, we are actually causing danger
because maybe some other decision theory holds. But Yudkowski does

(01:01:41):
not think that's the case. Also, one more thing, it
depends on the basilist. So if you think this scenario
could be real, it depends on it not having ethical
or behavioral controls that would prevent it from engaging in torture. Yeah.
And I think if thinkers like the you know, the
Miria people, the Machine Intelligence Research Institute people, succeed in

(01:02:03):
established what they're trying to do is establish a philosophical
framework to make AI friendly, to make it so that
it is not evil and does not harm us. And
if they successfully do that, then this shouldn't be a problem,
right Because Yudkowski has argued that a being that tries
to do what's best for us would not engage in
torture and blackmail, even if it's doing so in service

(01:02:26):
of some higher good, because doing torture and blackmail are
actually not compatible with human values. I agree with that absolutely,
and I actually would go as far to say I
think that's something people should keep in mind when they're uh,
when they're they're choosing their religions as well. Yeah, I
can certainly see how you can make that argue. Yeah,
it's like, what do I love about my faith? Is

(01:02:48):
that the blackmail and the torture? Or is there something
that brings something else to the table that is worth
living forward that makes life better for everybody? Like, like
I feel like, is what should be important about one's faith? Now,
I think some people might be saying, like, wait a minute, though,
if you're just using utilitarian ethics, right, wouldn't wouldn't any

(01:03:11):
methods be good if the if the ends justify the means, right?
That that's I think a naive understanding of how people
think about utilitarian ethics. If you want to bring about
the greatest good for the greatest number of people, couldn't
you do that by being really cruel and unfair to
some smaller group of people? And I think generally there
are versions of utilitarianism that say, well, actually, the answer

(01:03:33):
there is no, you couldn't do that, because even though
you might be bringing about some better material circumstance, it
is actually corrosive to a society for things like that
to happen, even if they don't happen to many people. Right,
you say, what if I could make everybody on earth
ten percent happier on average by say, burying somebody in

(01:03:56):
a in a pit of bananas once a year, so
that you buried to death with bananas. Even the people
who are being made happier could very easily look at
that and say that's not fair and it makes the
world worse and I don't want it. And thus that
actually would be a subjectively relevant state. So we've talked
about AI risk on the show before, and you know,

(01:04:18):
one thing I feel like I still have not been
able to make up my mind about, despite reading a
lot on the subject, is that I don't know whether
it makes sense to take um to be super worried
about AI super intelligence and the risks associated. I mean,
I do think it's worth taking seriously and thinking about.
And I think people who want to devote their attention
to how, you know, how dealing with the control problem

(01:04:41):
and how you would get an AI to do things
that were good for us and not harmful to us,
that that's fine work. And I don't ridicule the people
who work on that problem the way some people do.
But on the other hand, I worry if by focusing
exclusively on sort of the machine god the super intelligence,
were sort of ignoring um much more plausible and current

(01:05:06):
threats that the ways that AI is already very plausibly
in a position to hurt us today or in the
very near future, and not depending on any outlandish assumptions.
The way it's already and will soon be used as
a cyber war weapon. The way it's hijacking our attention
and manipulating our opinions and behavior through social media and devices.

(01:05:27):
This is some of what our Scott Baker talked about
with his fears about AI. You don't actually need super
powerful AI to do a lot of damage. It just
needs to manipulate us in just the right kind of ways.
So not the great basilisks so much as all the
little basilisks that are out there, the little grass snakes,
grass with the tiny crowns. They can do a lot

(01:05:48):
of damage. And again I just want to be clear,
I'm not saying we should forget about super intelligence. People
who are working on that. If you find that interesting
that I think that's fine, Yeah, work on that problem.
But there but I think it's a longer shot, and
there's a lot of current and near future AI threat
that is really worth taking very seriously. I wish people
more people were devoting their lives to say AI cyber

(01:06:10):
weapons that are in development right now. One last issue
I think we should discuss before we wrap up here
is Okay, so we don't think this potential information hazard
is actually an information hazard, Like we don't think it's
actually potentially that dangerous. But Yudkowski has made the point
that even though he doesn't think the basilisk is plausible,

(01:06:32):
the ethical thing to do with potential information hazards is
to not discuss them at all, since it's possible that
they may be. Maybe maybe you're misinterpreting the ways in
which they're implausible. Maybe this idea is actually valid, is
actually relevant, and by spreading it you've harmed a lot
of people. But I also think that this could mean

(01:06:53):
that it's it's possible that despite the basilisk not being plausible,
something good has come out of the basilist conversation because
it encourages people to think of the idea of information hazards.
Maybe Rocco isn't true, but there could be other ideas
that are both true and potentially harmful to people. Just

(01:07:14):
by entering their minds. And the lesson from this is
we should prepare ourselves for those kinds of ideas. And
if you have discovered one of those ideas and there
is literally no upside to other people knowing about it,
keep it to yourself and don't post it on the internet. Well,
I feel like I do encounter thought hazards like this
from time to time that they're often presented in pamphlets

(01:07:35):
or little booklets, uh, generally with you know, a clever
illustration about the coming into the world. Actually brought some
of these into the office recently. I found them at
a park in rural Georgia, and uh, and I think
I told you it's like, Uh, you have have a
look at these. You may find them interesting, but UH,
do destroy them when you're done, because you know I did.

(01:07:57):
I did in the wrong hands. These thoughts can be
dangerous if they have some sort of a like a
harmful view of society that that people may buy into. Well,
I think you were comfortable sharing uh, malicious religious literature
with me because you do not think there's a possibility
that that literature is true and would harm me if

(01:08:18):
I knew it was true, like you think it is false,
So to you, it's actually not an information hazard. It's
just like an idea hazard. Uh. The real crazy thing
would be if you came across a pamphlet and you
read it and it's the equivalent of this raving malicious
religious literature, except you were convinced it was correct, if

(01:08:38):
it was more like that ring video I brought you exactly.
That is one of the things I've often seen on
the internet, this idea compared to the ring. But you know,
on the other hand, I do have to I am reminded,
you know that like the idea that that any kind
of knowledge is forbidden or is secret, like that doesn't
really jive well with just the the general mission of science,

(01:09:01):
of course not yeah, but I mean that would be
part of the problem of like we're not prepared for
information hazards, right, because in the past it's been the
case that almost anything that's true is good to spread, right,
unless you're spreading lies, information is good to share. It's
just possible we should acknowledge that maybe there is such
a thing as a fact that or a fact or

(01:09:22):
an idea or a theory or something that is true
and correct. But it would hurt people to know about it.
I can't think of an example of anything like that,
but if there is something like that, we we should
be ready to not spread it when it occurs to us.
All right, fair enough, Well, I want to close out
here with just a one more bit of basilisk wisdom

(01:09:43):
or anti basilist wisdom. And this comes from the poetry
of Spanish author Francisco Gomez de Gravado e Viegas, who
Live five. This is translated and it's referenced in Carol
Rose's Giants, Monsters and Dragons quote. If the person who
saw you was still living, then your whole story is lies,

(01:10:03):
since if he didn't die, he has no knowledge of you,
and if he died, he couldn't confirm it. So I
was thinking about that with the stories of the basilist. Yeah,
I was like, wait a minute, that How would you
know if you could die just by looking at something?
How do we have this description in the book? Yeah,
it's uh, there is a there's an authorship problem with this. Yeah,

(01:10:26):
whose whose account is the basilist? But at any rate,
I think it's a nice like final you know, sucker
punch to the basilists in general, but also a little
bit to the idea of the great basilisk, right. I
hope you were not leaving this episode with with terrors
about future digital torment. I think that is not something
that you should worry about. Indeed, I'm not worried about it,

(01:10:46):
and instead of worrying about it yourself, you should head
on over to stuff to Blow your Mind dot com.
That's the mother ship where you'll find all the podcast
episodes links out to our various social media accounts. Who
find the tab for our store, you can look up
that bassilist shirt design we're talking to out and uh
and that's a great way to support the show. And
if you don't want to support the show with money,
you can do so by uh simply rating and reviewing

(01:11:08):
us wherever you have the power to do so. Big
thanks as always to our wonderful audio producers Alex Williams
and try Harrison. If you would like to get in
touch with us to let us know feedback on this
episode or any other, to suggest a topic for the future,
or just to say hi, let us know whether you
carry a weasel around In case of a basilisk encounter,
you can email us at Blow the Mind at How

(01:11:30):
Stuff Works dot com. Oh, I'm hearing that transmission again.
That weird snow. What is that? Oh Hell, the Great Bassilisk.
All Hell, the Great Basilisk. Oh Hell the Great Bassilist.
All Hell, the Great Bassilist. Hell. There the great basilists
call Hell the Great Basilisk, Hell the Great Bassilisk. O Hell,

(01:11:57):
the Great Basilisk,