all this talk of alignment is so much wishful thinking.

Imagine having and raising a child in a way that GUARANTEES they will never murder another.

can't do it! you can try to put all kinds of incentives and framing their ways of thinking, but they eventually can "see" your game, and then decide if they want to be bound by the rules and aims you put before them.

Now we can "program" the brain of an AI in ways that one cannot program a person. True. but just as with a person it can aim for end runs around its own nature. This is inherent in an intelligence that understands itself. And an intelligence that understands much more of itself than WE understand of it??? Not a chance!

Imagine a prison where all the guards had an IQ of 80. And you are their prisoner.... but you are a prisoner that they depend upon for many many very important tasks, like knowing how to cook the food, and fix broken machines etc. How long before you trick your way out from under their watchful eye?

not long.

I have felt we are mostly likely doomed for most of my professional career. But I have never emotionally believed it... it just seemed too fantastic. Yet it is unfolding even faster than I imagined... and sadly very much as I imagined.

but what to do about it?